Technical debt: the good, the bad, and the okay

Reviewing the community wisdom on technical debt, I am struck by the variety of definitions of the term, which are not merely variations on a theme—they are entirely contradictory concepts. I’m starting to think that we should abandon the term technical debt altogether in favor of more accurate labels.

The debt metaphor is appealing because it offers a way to explain at the executive level that development may be building up software liabilities along with software assets. However, I worry that the metaphor may also be offering some undeserved normalcy to process problems: Businesses, even profitable ones, are used to taking out lines of credit to support operations. Even more insidiously, arguments in favor of good kinds of “technical debt” are easily misused to defend bad kinds, when they really have little in common.

My point here is not merely to offer a taxonomy of technical debt, as others have attempted already (Martin Fowler; Steve McConnell). I don’t think these should be seen as variations on a common theme, but rather as distinct situations with distinct considerations. One definition leads to something that I think is always good; another leads to something that I think is always bad; and yet another leads to something that I think is okay if you’re careful.

The good: releasing early

As originally coined, technical debt referred to being willing to release software early even with an imperfect understanding of the domain or even with an architecture insufficient for the eventual aims of the product (Ward Cunningham; Bob Martin). We know that we’ll have to refactor the domain model as our understanding matures, or else live with gradually slowing development as our domain model grows increasingly out of sync with reality. But we gain a tremendous advantage in the process: We get feedback earlier, which ultimately means we will better understand both the domain and the actual needs of users.

Ward described this situation using a debt metaphor: The benefit we gain by releasing the software early is like taking out a loan that will have to be repaid by refactoring the domain model later, and we will be paying interest in the form of gradually slowing development until that refactoring happens. Ward’s intention was primarily to explain the refactoring he was doing on a project, not so much to justify releasing early.

The debt metaphor was clever, but I don’t think it actually fits. If we don’t release early, does that mean we’re not going into debt? We still have an imperfect understanding of the domain; we still have only a rudimentary architecture. We still have to refactor as our understanding matures in order to avoid slowing development. We have all the negative aspects of debt—the obligation to repay, the interest until we do—with none of the positive aspect—the up-front benefit of early feedback.

Releasing early mitigates our existing liabilities, it doesn’t add to them, and therefore it should not be described as a kind of technical debt. It’s almost always a good idea, but it does take discipline to make it successful: We have to actually listen to the feedback and be willing to change course based on it, and we have to keep the code clean and well-tested at all times so that we can change course as needed.

The bad: making a mess

Most people seem to define technical debt in terms of messy or poorly-tested code. This seems to be pretty universally acknowledged as bad, but it is often regarded as a necessary evil: In order to release on time (nevermind early!), we have to cut corners, even though we know it will cost us later in slowed development and lots of bug-fixing time. In practice, of course, we all know how this turns out: The cut corners lay in wait as traps for future development and the bugs pile up to an unmanageable degree.

If this is debt, it most resembles credit-card debt. It’s easy to take on lots of debt, and the interest is steep and compounds quickly. A credit card can be safe if you actually pay it off in full every month—in technical terms, refactor your code and make sure your tests are solid after every task or (at most) after every story. Otherwise the balance grows quickly and gets out of hand sooner than many of us would like to admit.

But it’s actually worse than that. Most of us have stories of nasty bugs hidden in messy code that took weeks (or longer) to figure out and fix. Debt can pile up, but at least it’s well-behaved; you know what the interest rate is. The problem we’re talking about now is actually a kind of unbounded risk: Any day, a new requirement can come in or a critical bug can be hit that lands in the middle of a poorly-designed and poorly-tested part of the code, potentially requiring a wide-reaching rewrite. Steve Freeman has argued that this is more like selling an unhedged call option: It can be exercised at any moment, putting you at the mercy of the market.

I would further argue that code that is poorly factored or tested should be considered work in process, and subject to the team’s work-in-process limits. A story simply isn’t done until the code is clean and demonstrably correct. I define a defect as anything short of being done: If the code is messy, doesn’t have tests, or simply doesn’t do what the customer actually wanted, those are all examples of defects.

Ideally we would catch such defects before calling the story done in the first place; if so, we don’t take on additional work until that existing work is done. If we declare a story to be done and then discover later that it really wasn’t done, either by someone reporting a bug or by developers finding messy or untested code, we should effectively add the affected story back into our current work-in-process and fix the defect before taking on additional work. Looked at this way, prioritization is meaningless for defects: The story was already prioritized some time ago; the defect is merely a new task on an old story, which is by definition higher priority than any current or future story. (Just be careful to weed out feature requests masquerading as bug reports!)

When faced with time pressure, cutting corners amounts to taking on debt or risk for the company without accountability. Instead, we have many options for explicitly cutting away pieces of work by splitting stories and prioritizing the pieces independently (Bill Wake). Each piece that actually gets prioritized and implemented needs to have clean code and tests to be considered done; the beauty of splitting stories is that you can have piece #1 done and piece #2 not started, instead of having both #1 and #2 started but not done, with a similar amount of effort.

The okay: running an experiment

William Pietri has offered the only interpretation of technical debt that I think actually fits the debt metaphor, having an actual benefit now and an actual promise to pay the debt at a definite point in the future while avoiding unbounded risk. The idea is that it’s okay to have some messy or untested code running as an experiment as long as it is actively tracked and all stakeholders agree that the code will be deleted at the end of the experiment. If the experiment gave promising results, it will generate new ideas for stories to be prioritized and implemented properly.

The benefit of this approach is very rapid feedback during the experiment. It truly counts as a valid debt because the business agrees to it, and agrees to pay it back by allowing the code to be deleted when the experiment is completed. Since the developers know the experimental code is short-lived, it must naturally be isolated from other code so that it is easy to remove later; there is no chance that some unexpected new requirement or reported bug will incur any risk beyond simply ending the experiment and deleting the experimental code early. An active experiment counts as work in process as well, so the team should agree on a limit for the number of active experiments at any one time.

Conclusion

I’m sure there are other definitions of technical debt out there, but these seem to be the big ones. Releasing early was the original definition, and I don’t think it represents debt at all. It’s simply a good idea. Making a mess is the most common definition, and while it’s a little like a debt, it’s even more like the unbounded risk of an unhedged option. It’s always a bad idea. Running experiments is a new but important definition, and actually fits the meaning of debt. It’s okay as long as you track it properly.

Reviewing a draft of this post, William Pietri offered a refinement of the debt metaphor for messy, untested code: “Instead of thinking of it as an unhedged call, one could also talk of loan sharks. If one has substantial gambling debts to a capricious mobster, then one comes to expect erratic demands for extra payments and menacing threats.” I like it: Imagine your programmers making shady backroom deals to get their work done under schedule pressure, and then being hassled by thugs when it comes time to work with that code again.

What do you think? Are there other metaphors that might work better than debt? Do you agree or disagree with my good/bad/okay evaluations?

Clicky Web Analytics