AI-Assisted Refactoring of Inherited Technical Debt

Here's the thing nobody tells you when you inherit a codebase: the debt isn't static. It's accruing interest while you're still trying to figure out where the bathroom is.

McKinsey has found that technical debt can consume 40 to 50 percent of total IT investment, yet it almost never appears as a named line item in any budget conversation. It hides inside maintenance costs, delayed features, and the slow attrition of engineers who quietly stop caring about the system they work in. A separate 2025 McKinsey report found organizations spending 60 to 80 percent of IT budgets just keeping legacy systems alive. IBM puts the revenue-generating portion of the average tech budget at 23 percent. Everything else is treading water.

Developer cost compounds this further. CISQ and U.S. Department of Labor data puts the cost of fixing old code at roughly $3.60 per line, a figure that climbs as median programmer wages have grown more than 15 percent over five years. Stack Overflow's 2024 survey found developers spending over 17 hours per week on maintenance work. That's nearly half a working week, every week, not building anything new.

The AI-generated MVP handoff accelerates the compounding in a specific and underappreciated way. The code looks fine. It passes a code review on the surface. But GitClear's 2025 analysis of 211 million lines of code found that code churn in AI-assisted repositories climbed from 3.1 percent in 2020 to 5.7 percent in 2024. Churn, in this context, is code that gets written and then rewritten quickly, which is the fingerprint of structural debt accumulating in real time, not a one-time problem you fix and forget.

And then there's morale. Sixty-three percent of professional developers in Stack Overflow's 2024 survey cited technical debt as their top frustration. Teams working in inherited debt don't just lose time; they lose people. The engineers who have options exercise them.

So the question is not whether to address the debt. It's whether you can make a meaningful dent without stopping the product entirely. That's where the real work begins.

The Triage Mindset: Why Refactoring Inherited Debt Requires a Different Approach Than Greenfield Work

I want to be precise about what "triage" actually means here, because it gets used loosely.

Triage is not about finding the ugliest code and cleaning it up. It is systematic prioritization: deciding what to understand, what to stabilize, and what to change, in that order, before touching a single line of production code. The constraint that makes inherited codebases categorically different from greenfield work is this: you cannot afford to break behavior you haven't mapped yet.

In new code, a breaking change is usually recoverable. You know what you intended. In a production system with real users and undocumented logic, a silent behavior change can travel through the system for weeks before surfacing as a customer complaint or, worse, a data integrity problem.

The mental model that has stuck with me is the strangler fig. Rather than replacing the running system wholesale, you modernize it incrementally alongside it, wrapping new behavior around old functionality in stages until the old core can be retired safely. The reason this model matters is not aesthetic. It's structural: it forces you to work in bounded increments, which is exactly the discipline that prevents refactoring from becoming its own source of chaos.

The work divides into three phases that must occur in sequence. First, understanding: mapping what the system actually does, its dependencies, its data flows, its undocumented logic. Second, stabilization: building a test harness around existing behavior before any of it changes. Third, refactoring: making incremental, verifiable changes with each step confirmed by the test suite. AI belongs in all three phases, but doing different jobs in each. And one rule runs through all of it without exception: AI proposes, humans approve.

Using AI to Build a Map of a Codebase You Didn't Write

Before AI, reverse-engineering business logic from undocumented legacy code was weeks of senior-engineer time, poring over procedures, tracing call stacks, interviewing whoever hadn't left the company yet. Language models compress that substantially, not to zero, but often from weeks to days.

In the understanding phase, you are asking AI to do several specific things. Generate plain-English explanations of functions, classes, and modules, especially legacy logic with no comments. Produce inline documentation across the codebase. Map dependency graphs: which modules call which, where shared state lives, what the blast radius is if a given function changes. Surface anti-patterns: circular dependencies, god classes, implicit coupling between components that were never meant to be coupled.

The model you use matters less than the specificity of your prompts. GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and code-specialized models like StarCoder2 are all applicable; the right choice depends on your stack and context window requirements. Tools like CodeScene add a layer that raw LLMs don't provide: change frequency and defect correlation data, which tells you not just where the problems are but which problems are actually causing the most pain right now.

The output of this phase is not cleaned code. It is a prioritized inventory: which areas carry the most change-related risk, where the undocumented business logic is concentrated, and what can be safely isolated versus what is load-bearing infrastructure that everything else depends on.

One guardrail that cannot be skipped: AI explanations of inherited code can be wrong in ways that sound entirely confident. This isn't a hypothetical. The model has no access to the business context that shaped the original decisions. It is pattern-matching against what code like this usually does, not what this code specifically does. Treat all generated documentation as a draft requiring engineer review. The reason this matters becomes concrete in the next phase: a test harness written around incorrect documentation will protect the wrong behavior.

Building a Test Harness Before Refactoring Anything

The logic here is simple and unforgiving. You cannot safely change behavior you have not characterized. Without a test harness, refactoring is just editing with optimism.

The inherited-code test problem is that tests typically don't exist, and writing them manually against undocumented logic is slow and prone to encoding assumptions that turn out to be wrong. This is why the mapping phase has to come first. You cannot write meaningful tests around behavior you haven't confirmed you understand.

The AI-assisted approach works like this: feed the model the annotated module documentation from phase one, along with actual function signatures and any runtime examples you can capture. The model generates candidate unit and integration tests covering the documented behaviors. An engineer reviews those tests for correctness and fills the gaps where the model has misread edge cases or missed conditional paths.

What the resulting test suite actually does is create a regression baseline. Any refactoring change that breaks a passing test is caught immediately. This is what makes the "silent behavior change" risk visible and manageable rather than latent and terrifying. It's also what enables the strangler-fig approach to work in practice; you need confidence at each small step, not just at the end.

The guardrail remains: AI-generated tests can encode incorrect behavior if the phase-one documentation was itself inaccurate. Engineer sign-off on test logic is mandatory before you use that suite as a safety net for anything. I've seen teams skip this review step because the generated tests looked reasonable, and "looked reasonable" turned out to be doing a lot of work.

Running Incremental Refactors With AI as a Pattern-Execution Tool, Not a Decision-Maker

With a map and a test harness in place, you can begin changing things. What AI does well in this phase is mechanical, pattern-based transformation executed consistently across a large codebase: renaming, extracting functions, flattening deeply nested conditionals, removing duplicated logic. These tasks are tedious and error-prone for humans to do at scale; they are well within the capabilities of a language model following a clear prompt.

What AI should not do in this phase is make architectural decisions. Which services to split, where to draw domain boundaries, what to deprecate: these are judgment calls requiring understanding of the business, the team's capacity, and the actual failure modes of the system. The model doesn't have that understanding, regardless of how confident its proposals appear.

The workflow that keeps this phase safe has a specific rhythm. AI proposes a scoped, specific change, one function, one module, one pattern. The engineer reviews the diff, small enough to read and understand in a single sitting. The test suite runs. If tests pass, the change is committed. If they don't, the proposal is rejected and the engineer investigates. Then you repeat. The refactoring is a sequence of small, verifiable steps, not a large branch sitting open for two weeks while everyone argues about it.

The anti-pattern to avoid is using AI to generate a large rewrite of a problematic module in one pass. This recreates exactly the conditions that produced the inherited debt in the first place: opaque, unreviewed code entering the codebase without anyone fully understanding what it does. GitClear's 2025 research found that refactoring as a share of developer activity dropped from roughly 25 percent in 2021 to less than 10 percent in 2024 as AI adoption increased. Teams appear to be using AI to write more code rather than to improve the code they have, which is precisely the opposite of what this problem requires.

Treat each AI-assisted refactoring session as a unit of work that must be completable and reviewable before the next one begins.

Where the AI-Assisted Workflow Breaks Down and What to Do About It

The hallucination problem is the most insidious failure mode, and I want to be specific about why.

Large language models generate confident explanations of code that can be wrong in subtle, domain-specific ways, especially for legacy business logic with no documentation to cross-check against. The danger is not a single wrong answer. It's that an incorrect explanation in phase one propagates forward through the entire workflow: wrong documentation produces wrong tests, and wrong tests allow refactoring that breaks real behavior while the suite continues to pass. By the time you find it, you have committed months of work on top of a flawed foundation.

Context window limits create a related problem on large codebases. Even 2025-generation models with generous context windows struggle with dependencies that span many files, because the model's picture of the system is necessarily partial. The practical mitigation is to feed the model one module at a time, with its direct dependencies explicitly included in the prompt. Do not assume the model has a coherent picture of the whole system; it almost certainly doesn't.

The new-debt risk is real when the workflow is followed loosely. Harness's State of Software Delivery 2025 found 67 percent of developers reporting that they spend more time debugging AI-generated code than they expected. That cost lands squarely on the refactoring team if they accept AI output without scrutiny. Ox Security's analysis of 300 repositories found that duplication and avoidance of abstraction were among the most consistent anti-patterns in AI-generated code. A loose review process will import those patterns directly into the codebase you're trying to improve.

There are also categories of judgment that AI cannot substitute for, and this is worth sitting with. The most consequential is distinguishing behavior that is a bug from behavior that is a feature from behavior that is a regulatory requirement. That distinction lives in institutional knowledge, product history, and sometimes legal context; none of it is derivable from the code itself. When you ask an AI to explain why a piece of legacy logic works the way it does, it will give you a structurally plausible answer. Whether that answer reflects the actual reason requires a human who remembers, or can find out.

How to Decide Where to Start When the Whole Codebase Is a Problem

When AI maps a codebase and surfaces dozens of problem areas simultaneously, the instinct is to feel like you now understand the problem. You don't. You have a list. Understanding comes from deciding which problems actually matter first, and that requires different reasoning than the mapping did.

Two dimensions should govern the starting point, held in tension with each other.

The first is change frequency. Code that is touched most often is causing the most daily friction: breaking most often, consuming the most debugging time, creating the most merge conflicts. Start here, not with the file that is architecturally most offensive but rarely touched. An elegant refactor of code nobody uses accomplishes nothing except making the person who wrote the refactor feel good.

The second is defect correlation. Areas where bugs consistently cluster, regardless of how frequently the code changes, indicate structural problems rather than just complexity. These are the places where the architecture is actively misleading engineers, where the implicit assumptions of one module are invisible to the module depending on it. CodeScene surfaces both dimensions in combination, showing you which files sit at the intersection of high change frequency and high defect concentration. That intersection is your starting point.

One additional filter before committing to a target: reversibility. Prefer changes that can be made in small, isolated steps and rolled back if something goes wrong. Avoid starting with components so deeply load-bearing that any change to them requires coordinating a dozen other changes simultaneously. Build momentum with wins that are verifiable before moving to the parts of the system that require months of careful work to touch safely.

The teams that make real progress on inherited debt are the ones that resist the urge to fix everything at once and instead build a credible sequence: one area, fully characterized, tested, and incrementally improved, before moving to the next. That sequencing is not administrative overhead. It is the work. AI can accelerate every stage of it. It cannot substitute for the judgment that decides what the sequence should be, and if you hand that judgment to the tool, you will eventually find yourself inheriting a new problem that looks exactly like the old one.