Technical Debt

  1. Posted on November 1, 2007 1:06:PM by Steve McConnell to 10x Software Development
  2. Technique, executives, Non-Technical Stakeholders, Technical Debt, Design

The term technical debt was coined by Ward Cunningham to describe the obligation that a software organization incurs when it chooses a design or construction approach that's expedient in the short term but that increases complexity and is more costly in the long term.

Ward didn't develop the metaphor in very much depth. The few other people who have discussed technical debt seem to use the metaphor mainly to communicate the concept to technical staff. I agree that it's a useful metaphor for communicating with technical staff, but I'm more interested in the metaphor's incredibly rich ability to explain a critical technical concept to non-technical project stakeholders.

What is Technical Debt? Two Basic Kinds

The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job. In some cases, this kind of debt can be incurred unknowingly, for example, your company might acquire a company that has accumulated significant technical debt that you don't identify until after the acquisition. Sometimes, ironically, this debt can be created when a team stumbles in its efforts to rewrite a debt-laden platform and inadvertently creates more debt. We'll call this general category of debt Type I.

The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future. "If we don't get this release done on time, there won't be a next release" is a common refrain—and often a compelling one. This leads to decisions like, "We don't have time to reconcile these two databases, so we'll write some glue code that keeps them synchronized for now and reconcile them after we ship." Or "We have some code written by a contractor that doesn't follow our coding standards; we'll clean that up later." Or "We didn't have time to write all the unit tests for the code we wrote the last 2 months of the project. We'll right those tests after the release." (We'll call this Type II.)

The rest of my comments focus on the kind of technical debt that's incurred for strategic reasons (Type II).

Short-Term vs. Long-Term Debt

With real debt, a company will maintain both short-term and long-term debt. You use short-term debt to cover things like gaps between your receivables (payments from customers) and expenses (payroll). You take on short term debt when you have the money, you just don't have it now. Short-term debt is expected to be paid off frequently. The technical equivalent seems straightforward. Short-term debt is the debt that's taken on tactically and reactively, usually as a late-stage measure to get a specific release out the door. (We'll call this Type II.A.)

Long term debt is the debt a company takes on strategically and proactively--investing in new capital equipment, like a new factory, or a new corporate campus. Again, the technical equivalent seems straightforward: "We don't think we're going to need to support a second platform for at least five years, so this release can be built onthe assumption that we're supporting only one platform." (We'll call this Type II.B.)

The implication is that short-term debt should be paid off quickly, perhaps as the first part of the next release cycle, whereas long-term debt can be carried for a few years or longer.

Incurring Technical Debt

When technical debt is incurred for strategic reasons, the fundamental reason is always that the cost of development work today is seen as more expensive than the cost will be in the future. This can be true for any of several reasons.

Time to Market. When time to market is critical, incurring an extra $1 in development might equate to a loss of $10 in revenue. Even if the development cost for the same work rises to $5 later, incurring the $1 debt now is a good business decision.

Preservation of Startup Capital. In a startup environment you have a fixed amount of seed money, and every dollar counts. If you can delay an expense for a year or two you can pay for that expense out of a greater amount of money later rather than out of precious startup funds now.

Delaying Development Expense. When a system is retired, all of the system's technical debt is retired with it. Once a system has been taken out of production, there's no difference between a "clean and correct" solution and a "quick and dirty" solution. Unlike financial debt, when a system is retired all its technical debt is retired with it. Consequently near the end of a system's service life it becomes increasingly difficult to cost-justify investing in anything other than what's most expedient.

Be Sure You Are Incurring The Right Kind of Technical Debt

Some debt is taken on in large chunks: "We don't have time to implement this the right way; just hack it in and we'll fix it after we ship." Conceptually this is like buying a car—it's a large debt that can be tracked and managed. (We'll call this Type II.A.1.)

Other debt accumulates from taking hundreds or thousands of small shortcuts--generic variable names, sparse comments, creating one class in a case where you should create two, not following coding conventions, and so on. This kind of debt is like credit card debt. It's easy to incur unintentionally, it adds up faster than you think, and it's harder to track and manage after it has been incurred. (We'll call this Type II.A.2.)

Both of these kinds of debt are commonly incurred in response to the directive to "Get it out the door as quickly as possible." However, the second kind (II.A.2) doesn't pay off even in the short term of an initial development cycle and should be avoided.

Debt Service

One of the important implications of technical debt is that it must be serviced, i.e., once you incur a debt there will be interest charges.

If the debt grows large enough, eventually the company will spend more on servicing its debt than it invests in increasing the value of its other assets. A common example is a legacy code base in which so much work goes into keeping a production system running (i.e., "servicing the debt") that there is little time left over to add new capabilities to the system. With financial debt, analysts talk about the "debt ratio," which is equal to total debt divided by total assets. Higher debt ratios are seen as more risky, which seems true for technical debt, too.

Attitudes Toward Technical Debt

Like financial debt, different companies have different philosophies about the usefulness of debt. Some companies want to avoid taking on any debt at all; others see debt as a useful tool and just want to know how to use debt wisely.

I've found that business staff generally seems to have a higher tolerance for technical debt than technical staff does. Business executives tend to want to understand the tradeoffs involved, whereas some technical staff seem to believe that the only correct amount of technical debt is zero.

The reason most often cited by technical staff for avoiding debt altogether is the challenge of communicating the existence of technical debt to business staff and the challenge of helping business staff remember the implications of the technical debt that has previously been incurred. Everyone agrees that it's a good idea to incur debt late in a release cycle, but business staff can sometimes resist accounting for the time needed to pay off the debt on the next release cycle. The main issue seems to be that, unlike financial debt, technical debt is much less visible, and so people have an easier time ignoring it.

How do You Make an Organization's Debt Load More Visible?

One organization we've worked with maintains a debt list within its defect tracking system. Each time a debt is incurred, the tasks needed to pay off that debt are entered into the system along with an estimated effort and schedule. The debt backlog is then tracked, and any unresolved debt more than 90 days old is treated as critical.

Another organization maintains its debt list as part of its Scrum product backlog, with similar estimates of effort required to pay off each debt.

Either of these approaches can be used to increase visibility into the debt load and into the debt service work that needs to occur within or across release cycles. Each also provides a useful safeguard against accumulating the "credit card debt" of a mountain of tiny shortcuts mentioned earlier. You can simply tell the team, "If the shortcut you are considering taking is too minor to add to the debt-service defect list/product backlog, then it's too minor to make a difference; don't take that shortcut. We only want to take shortcuts that we can track and repair later."

Ability to Take on Debt Safely Varies

Different teams will have different technical debt credit ratings. The credit rating reflects a team's ability to pay off technical debt after it has been incurred.

A key factor in ability to pay off technical debt is the level of debt a team takes on unintentionally, i.e., how much of its debt is Type I? The less debt a team creates for itself through unintentional low-quality work, the more debt a team can safely absorb for strategic reasons. This is true regardless of whether we're talking about taking on Type I vs. Type II debt or whether we're talking about taking on Type II.A.1 vs. Type II.A.2 debt.

One company tracks debt vs. team velocity. Once a team's velocity begins to drop as a result of servicing its technical debt, the team focuses on reducing its debt until its velocity recovers. Another approach is to track rework, and use that as a measure of how much debt a team is accumulating.

Retiring Debt

"Working off debt" can be motivational and good for team morale. A good approach when short-term debt has been incurred is to take the first development iteration after a release and devote that to paying off short-term technical debt.

The ability to pay off debt depends at least in part on the kind of software the team is working on. If a team incurs short-term debt on a web application, a new release can easily be rolled up after the team backfills its debt-reduction work. If a team incurs short-term debt in avionics firmware— the pay off of which requires replacing a box on an airplane— that team should have a higher bar for taking on any short-term debt. This is like a minimum payment--if your minimum payment is 3% of your balance, that's no problem. If the minimum payment is $1000 regardless of your balance, you'd think hard about taking on any debt at all.

Communicating about Technical Debt

The technical debt vocabulary provides a way to communicate with non-technical staff in an area that has traditionally suffered from a lack of transparency. Shifting the dialog from a technical vocabulary to a financial vocabulary provides a clearer, more understandable framework for these discussions. Although the technical debt terminology is not currently in widespread use, I've found that it resonates immediately with every executive I've presented it to as well as other non-technical stakeholders. It also makes sense to technical staff who are often all-too-aware of the debt load their organization is carrying.

Here are some suggestions for communicating about debt with non-technical stakeholders:

Use an organization's maintenance budget as a rough proxy for it's technical debt service load. However you will need to differentiate between maintenance that keeps a production system running vs. maintenance that extends the capabilities of a production system. Only the first category counts as technical debt.

Discuss debt in terms of money rather than in terms of features. For example, "40% of our current R&D budget is going into supporting previous releases" or "We're currently spending $2.3 million per year servicing our technical debt."

Be sure you're taking on the right kind of debt. Not all debts are equal. Some debts are the result of good business decisions; others are the result of sloppy technical practices or bad communication about what debt the business intends to take on. The only kinds that are really healthy are Types II.A.1 and II.B.

Treat the discussion about debt as an ongoing dialog rather than a single discussion. You might need several discussions before the nuances of the metaphor fully sink in.

Technical Debt Taxonomy

Here's a summary of the kinds of technical debt:

Non Debt

Feature backlog, deferred features, cut features, etc. Not all incomplete work is debt. These aren't debt, because they don't require interest payments.

Debt

I. Debt incurred unintentionally due to low quality work

II. Debt incurred intentionally

II.A. Short-term debt, usually incurred reactively, for tactical reasons

II.A.1. Individually identifiable shortcuts (like a car loan)

II.A.2. Numerous tiny shortcuts (like credit card debt)

II.B. Long-term debt, usually incurred proactively, for strategic reasons

Summary

What do you think? Do you like the technical debt metaphor? Do you think it's a useful way to communicate the implications of technical/business decision making to non-technical project stakeholders? What's your experience? I look forward to your thoughts.

Resources

bk said:

November 1, 2007 5:28:PM

Interesting way of distinguishing between innovation and maintenance work, but no one will ever remember the meaning of "type I debt," "type IIA debt," etc.  It"s better to label it more descriptively.

Rock Hymas said:

November 1, 2007 5:34:PM

I so needed to read this. I have definitely started leaning towards the stereotypical attitude of technical staff: "the only correct amount of technical debt is zero." This helps me to understand that there is business value in taking on certain kinds of technical debt. It"s also a great model that will allow me to discuss technical debt with my manager(s) in the future.

Steve McConnell said:

November 1, 2007 5:50:PM

OK, How about these labels for kinds of debt?

I. "Unintentional Debt." Debt incurred unintentionally due to low quality work

II. "Intentional Debt." Debt incurred intentionally

     II.A. "Short-Term Debt." Short-term debt, usually incurred reactively, for tactical reasons

        II.A.1. "Focused Short-Term Debt." Individually identifiable shortcuts (like a car loan)

        II.A.2. "Unfocused Short-Term Debt." Numerous tiny shortcuts (like credit card debt)

     II.B. "Long-Term Debt." Long-term debt, usually incurred proactively, for strategic reasons

Alejandro Garcia said:

November 1, 2007 6:42:PM

Type I is the:  "health bills debt."

In some cases you could probably make something to prevent it (like diet and exercise or training and better selection), but also sometimes it is completely unexpected (like when you break your leg, or your server melt)

Type II.B

is the "mortgage" debt. You do it voluntary, you know is big and it will take you a while to pay but, in the future you will have a big asset.

Richard Quinn said:

November 2, 2007 12:45:AM

Thanks for the article Steve, it really does help conceptualize a problem.

However, I think the concept of debt in this setting is misleading. Usually, debt is incurred because you have borrowed to spend now, which must be paid off tomorrow. In a software engineering setting what has been borrowed? What has been spent? The only concept which is clearly explainable is the idea of having to pay back over the coming weeks or months.

I predict, that if I talk to business people about technical debt the first thing they will ask me is: "Who authorized the loan? What was the extra money (time/staff/feature) spent on?". There is - I think - a clear gap here between the kind of debt software engineers must service and the kind which finance people must service, this gap may well lead to more, not less, understanding of the issue amongst non-technical business deciders.

Perhaps I missed something, I"m looking forward to your reply.

Big Maybe said:

November 2, 2007 8:50:AM

Richard,

You borrow Quality, in order to buy Time. Assuming 100% quality is the ideal, and requires no interest payments (in the form of future maintenance work that otherwise wouldn"t have materialized), then every bit of quality sacrificed now is "borrowed" in the sense that you will eventually pay it back, with interest, using time that you assume you will have in the future.

Steve McConnell said:

November 2, 2007 9:27:AM

Richard, I look at it about the same Big Maybe described. You"re acquiring an asset (the released software) for less than it"s full price. Thus you"re financing the gap between what you paid (actual dev cost) vs. the full price (what the dev cost would have been without the shortcuts). That gap is the technical debt.

My experience is that business people get the concept right away. I haven"t found that they asked questions like, "Who authorized the loan?" On the contrary--it puts a highly technical topic into a context that they understand, which makes them feel like they a better understanding of the issues. Of course they will ask lots of questions, but that"s to be expected in any dialog about technical/business tradeoffs.

bk said:

November 2, 2007 10:33:AM

For me, the most valuable take-aways here are the "How do You Make an Organization"s Debt Load More Visible?" and "Retiring Debt" sections.

If you actually put pen to paper and record the compromises you are making during your software development cycles, and then you get to the point where you see this list and say "ok, our list is big enough, time to work on making it smaller,"  then that kind of hard-accountability can have a real positive effect.  The challenge, as I see it, is figuring out what actually needs to go on that piece of paper to make it a valuable tool.  

Doug Sjoquist said:

November 2, 2007 11:03:AM

Type I/Unintentional debt occurs when making an impulse or other poorly thought out decision without realizing you are incurring debt.  It is similar to spending money from a checking account that isn"t there, so perhaps a reasonable financial analogy for non-technical people would be "overdrafts".  

Most of us have some overdraft protection on checking accounts to protect us against our own laziness or errors, but the tolerance for it is pretty low.  Similarly, most organizations can tolerate a little of this type without too much pain, but the curve rises steeply at a fairly early point with this type debt.

Richard Quinn said:

November 2, 2007 12:05:PM

Thanks to Big Maybe and Steve for the answers.

I had already understood that quality was being sacrificed in order to gain development time. The new concept introduced in the article is the idea that the quality is not really sacrificed, at some point later it will be added (with interest) - therefore it was just borrowed and not sacrificed. In any case, the premise of the article is to use a terminology familiar in a business setting. So here we are borrowing dollars which would have been spent on programming days now which should be paid back later. The thing that was purchased was the business benefit gained from using the delivered functionality n months sooner than otherwise.

In a financial setting, the questions would be: we borrowed money. What will happen if we default on the repayments? In the real world foreclosures will happen, and so the (monetary) debt really must be paid back to avoid foreclosure and other nasty events.

What is the "zwang", the necessity, to repay the quality debt? How does the threat of foreclosure transfer from the financial to the software engineering world?

Currently I am having difficulty justifying taking a development team of 20 core developers + ancillary, test and documentation staff away from programming (delivering business value) for three months in order to straighten out software which has been deteriorating for years. I have no bailiffs, sheriffs or jails with which I can threaten senior management with dire consequences for none-repayment of the quality debt. In previous years, and most probably int the near future these people have consistently chosen to borrow more instead of servicing debt.

Thanks for your time,

Richard

Steve McConnell said:

November 2, 2007 12:33:PM

Richard, No question that the analogy is going to break down at some point. The "zwang" is that the more debt you accumulate, the more your rate of progress (i.e., velocity) will slow down. Like business debt, there isn"t really an expectation that you *ever* pay it off completely. The goal is to keep the debt service at a reasonable level compared to your other expenses & investments.

I wouldn"t try to convince senior management to *pay off* the debt. I would try to convince them to *pay down* the debt (i.e., pay off part of it). There are several possible leverage points you can use with senior management:

* "We"ll be able to get future releases out in 5 months instead of 6 [or whatever] if we can spend X-amount of time paying down our technical debt."

* "We can add functionality in area X, which currently we can"t do, if we spend Y weeks working on the technical infrastructure. Once we build the infrastructure for X, it will also allow us to A, B, and C fairly easily, which we can"t do now without that infrastructure."  

* "Our response time on hot fixes is really slow because there are major sections of the code that have become so complex that no one knows how to make changes in those areas safely. We end up with long review, test, and debug cycles even for simple changes, which requires a lot of time to release even a simple hot fix. If we spend X time paying down the debt, we"ll be able to reduce our average hot fix time by Y."

* "Some of the "shortcuts" we"ve been taking are starting to become visible to the customer, and the number of customer-reported defects has been increasing. Until we pay off some of the debt, we"re going to have an increasingly difficult time assuring the quality of our system prior to release."

These are just a few examples of the kinds of indirect costs you get when you have a high technical debt load. If your level of technical debt is such that you can"t promise specific business benefits along these lines from paying off the debt, then it"s always possible that the debt isn"t worth paying off.

Another reaction I had to your comment is that I don"t think I"ve ever seen a case where diverting a staff of 20 for 3 months to focus on debt reduction makes sense. I"ve seen companies attempt such initiatives many times, and they nearly always end up being unfocused boondoggles that don"t produce enough business value to justify their time or cost. I would suggest instead breaking the debt reduction payments into much smaller pieces and then including some percentage of debt reduction work into the team"s normal work flow. Software teams will often tell you that "It will be more efficient to do the debt reduction all at once," but it really isn"t because doing it all at once removes the "business value grounding" from the work.

Stephane Grenier said:

November 2, 2007 1:48:PM

Great article! I recently wrote something similar: www.followsteph.com/.../developer-debt

One thing I think that really helps to mitigate this is the Agile process. It"s by no means a Silver Bullet, but what it does do is help bring technical/development debt to the surface on an ongoing basis. You can"t escape it.

A lot of people want to think Agile development is fly by the seat of your pants development, or that it"s a Silver Bullet. Not at all. It"s basically a strategy for bringing forward issues such as technical debt on an ongoing basis (as well as continually giving your client deliverables, etc.). It gives you the ability to continually decide what"s worth adding to your credit card. And because of that, if used improperly it can be pretty easy to nickel and dime"ing yourself to debt. It must be used carefully, like all other development processes.

But because of the power the Agile process does have in bringing forth all these issues on an ongoing basis, making you face them rather than shoving them under the carpet, I believe it can really help many teams. At the very least, you"re going to be forced to make the decisions of whether or not to take on the debt.

Robin Barooah said:

November 2, 2007 6:02:PM

I think that one of the reasons why developers prefer to avoid technical debt is that technical debt is not fungible the way financial debts are.

Putting that another way - financial success of the product won"t change the fact that the only way to pay off the technical debt is for the developers to do painstaking and laborious work - which hurts the developers personally, whereas a financial debt can simply paid off out of revenues without anyone"s job being worse.

This might further explain why business people are prepared to accept technical debt - they aren"t the ones who are going to have to pay it off.  The developers suffer real consequences because they aren"t learning or growing by having to rework code that they knew was being done to substandard quality in the first place.

It"s also hard to ensure that all of the technical debt is paid off.  There"s no easy way to get a balance, and the interest compounds at a very high rate.

Robin Barooah said:

November 2, 2007 9:42:PM

As a follow up to my last comment, and to echo something others have already said here:

There"s no way to pay off the design debt "for it"s own sake".  After all, if the system works, there"s no justification for changing it.  Also, there"s no clear way to identify what constitutes the debt, or when it is paid off.

As far as I can see, the only way to meaningfully pay off the debt is in the context of refactoring in order to allow new functionality to be built without having to implement workarounds or duplication.

Hitesh said:

November 3, 2007 1:24:AM

Hi Steve, Great post.

Richard, I guess we are borrowing time! And payback in spending more time in the future.

Ward Cunningham said:

November 4, 2007 12:58:PM

This analysis and discussion is consistent with and and adds to the debt metaphor I introduced in my 1992 experience report. In that report I used "consolidation" to refer to refactoring, a word that was not in common use at the time.

I would also sometimes explain refactoring to my management as follows: "We tried to add the feature to our application but found that there was no place for it. So we first made a place, and then added the feature there."

I now ask teams to produce new functionality and new opportunity with each iteration. This requires skills that were not valued until recently. It was simply too easy to "burn" opportunity to deliver features and running out of opportunity was assumed to be a natural conclusion to any project.

cjlotz said:

November 4, 2007 11:01:PM

Thanks for a great discussion!  In our team we created a wiki page of items that we view as technical debt.  The idea is to have it public to remind ourselves of the items that we feel need to be addressed in the future.  The items are mostly Type I and Type II.A.  Team members are encouraged to take up some of these items whenever the opportunity arises and we also try and address a bigger chunk of these items in an iteration of technical debt work immediately following a big release.

Items completed are marked as completed on the wiki page.  We find it quite encouraging to see the list of items completed together with the work that still needs to be done.

Dave R said:

November 5, 2007 4:08:PM

Interesting stuff as usual, Steve.

(Probably a multiple post, sorry....)

We are doing this at present, modifying database tables to change data types to accommodate future enhancements. It"s repetitive grot work, the developers hate it, and there"s no immediate benefit; as the risks of squeezing in small enhancements outweigh the political benefits.

The trade-off comes next low season when we have  an infinitely more flexible base to provide potentially much bigger payoff.

... and now I have a new metaphor with which to dazzle the CEO ;)

Chui Tey said:

November 6, 2007 4:12:AM

I"d like to suggest the following class of technical debt

 Low interest debt in high inflationary environment.

In high inflationary environment it is often good to borrow money to purchase assets rather than holding on to cash. Similarly, some classes of technical debt can be paid simply by Moore"s Law of inflating processing power.

Case in point is how Microsoft products have been panned for being memory hogs in their early days, but in the long run, it became a strategic advantage as hardware became cheaper.

Similarly, using a high level language in a project despite early performance problems is akin to making technical bets on how fast the underlying technology can catch up.

Another kind of technical debt is the zero interest debt.  This is where you make a hackish workaround that may never require to be fixed.

When thinking about technical debt, it is also necessary to talk about technical equity, and return on equity. I"m channeling Kent Beck on this one, where developers are prone to developing features which are not required. In which case, there is no return on the invested equity.

vb said:

November 6, 2007 5:45:AM

Hi,

It is interesting to what lengths people can go to describe what happens in companies which provide services.

Because it is software we are more alert and alive to the problem.

I feel that the same situation exists in all other skill oriented industries - the only difference is that it does not get written about..

Big Maybe said:

November 6, 2007 9:09:AM

Chui Tey: Both points are excellent insights.

Another common software metaphor borrowed from the financial world: taxes. Some engineers use the term "taxes" to refer to code that is needed to support "regulatory" requirements, such as interoperability, accessibility, power management, and other features that are imposed on ALL software written in a similar "tax jurisdiction" (such as the same OS [country], or corporate network [county]).

A lot of debt is incurred just to meet a system"s tax bill. Not to stretch these metaphors too far, in a similar situation in the real world, it is pretty bad sign when a business borrows to pay taxes.

I think that exploring the reaches of the software tax metaphor deserves a post of its own. Eh, Steve? How about it?

Steve McConnell said:

November 6, 2007 3:23:PM

Big Maybe, Some of our clients have used the "tax" metaphor. I"ve mostly heard it used to refer to overhead required to develop a component within a larger system. I.e., to create even a "hello world" program the tax burden to work with the other programs is 5 staff weeks--after that you can actually start adding functionality.

That makes sense to me. Beyond that, I"m not sure I see a lot more implications of that particular metaphor, but I"d be happy to have someone *else* do a blog posting on it ;-)

Nathan Henkel said:

November 7, 2007 10:29:PM

Steve,

As I"ve often found to be the case, your analysis is detailed and well thought-out. I have an observation and a question.

Observation:

You mention that what I"d call "pure debt-reduction" phases rarely turn out well. I"d agree with that, and I think the reason why is that it"s really difficult to tell if your improving software without some sort of validation of the improvement. I might make changes that make a section of the code _look_ a lot prettier to me, but if those changes don"t make the code more flexible, I haven"t gained anything, and may have made things worse. It"s usually hard to tell if you are actually making your code more flexible in the absence of a flexor--a new requirement or other required change. I think usually technical debt reduction is not an activity in itself, but a way of approaching your normal activities. If debt-reduction in a particular area is desired, managers would encourage programmers working in that area to pursue the most flexible solution, even if it takes longer, to spend extra time refactoring, etc.

Question:

How do you go about estimating the principle and the interest rate of the debt you"re taking on? These things are typically fairly explicit when taking on financial debt, but are not so easily determined when it comes to technical debt. It seems the principle would be the cost of undoing the hack and redoing it the right way, and the interest would be the anticipated cost of changing new code that is somehow bound to the hack. However, determining the interest seems pretty hard here. If the hack happens just before a release, I guess you don"t need to estimate the interest IF you can make your first priority paying off the debt in the case where the interest turns out to be too high (i.e. many features in the new release will be hard to implement with the hack in place). Otherwise, you have to know what requirements are coming down the pipe, and the design implications of those requirements to accurately assess the interest. Assuming you can actually do this, you also have to accept that the interest rate is variable, meaning periodic re-estimation, the estimated cost of which must be rolled into the interest rate.

All this isn"t to say that it"s impossible to do this effectively, but it"s clearly seems a lot harder to manage knowledgeably than financial debt. I think this may be one reason that programmers shy away from this kind of debt--they know, whether because they"ve thought about it or just experienced it, that this sort of debt is difficult to manage.

Steve McConnell said:

November 8, 2007 12:49:PM

Nathan, Great comments. I agree that estimating the interest payment on technical debt is challenging at best. I do think you can estimate the debt itself -- the "principal" on the debt, so to speak.

When you get to a point where you are debating taking on technical debt, there are always at least 2 possible paths, one of which is the "good but expensive" path and one of which is the "quick and dirty" path. You wouldn't take on the technical debt in the first place unless you perceived a difference in cost between the "good" path and the "quick" path. When you reach that decision point, you could estimate the good path and the quick path. That estimate will help inform which path you should choose at that moment.

Separately, you should also estimate how much it will cost to backfill the good path *after* you've already gone down the quick path. Backfilling the good path will typically be more expensive than just doing the good path in the first place because the work will include ripping out the quick code, making sure you didn't introduce any errors while doing that, then adding the good code and going through the normal test & QA processes. The "ripping out" part makes it cost more to implement the good path later than it would have cost to implement it in the first place. And of course you've already incurred the cost of the quick path, so the real cost is the sum of the quick path + the clean path + the cost to rip out the quick path.

If the code is really well designed the "ripping out" cost can be minimal, but I think that's the exception.

The "interest payment" is trickier and depends very much on the specific case. I don't think there always is an ongoing interest payment. Sometimes the "interest" is really just the cost of ripping out the quick code and of implementing the good code. Other times the quick and dirty approach does create ongoing interest payments by making other related work take longer.

I think this might be a case where *controlling* this aspect of the project is a better choice than *estimating* it. I might encourage teams looking at these decisions to try to identify the cost to implement a quick and dirty path that doesn't affect other parts of the system (i.e., that doesn't create any ongoing interest payments) and also identify the cost of a quick and dirty path that does affect other parts of the system. So the decision table could look something like this example:

Option 1

Immediate cost of Good Solution: 10 staff days
Deferred cost to retrofit Good Solution: 0 staff days

Option 1 cost now: $6,000
Option 1 cost later: $0
Option 1 total cost: $6,000

Option 2

Immediate cost of Quick & Dirty solution with no interest payment: 3 staff days
Deferred cost to retrofit Good Solution: 12 staff days

Option 2 cost now: $1,800
Option 2 cost later: $7,200
Option 2 total cost: $9,000

Option 3

Immediate cost of Quick & Dirty solution with possible interest payment: 2 staff days
Deferred cost to retrofit Good Solution: 12 staff days
Estimated cost of "interest payments": 1-3 staff days

Option 3 cost now: $1,200
Option 3 cost later: $7,800-$9,000
Option 3 total cost: $9,000-$10,200

Travis Jensen said:

November 8, 2007 5:21:PM

This was a great post, Steve, and something I"ve been trying to quantify for some time.  On some things, calculating cost is easy: I have to keep paying person Y to do function X because we are using system Z.  In other cases, it is harder.  

I do want to think a bit more about the notion of debt and how it really plays in agile development, where doing things the simplest way is not a negative.  Certainly, in that context, the most painful debt is failure to refactor.  Many times, we just "hack it in" because we don"t want to pay for the refactor, but then we"ve introduced a pain point that will only grow.

Interesting topic, and undoubtedly one I will write more on at my blog as well (shameless plug: http://cmssphere.blogspot.com :).

Travis

Aaron Erickson said:

November 12, 2007 10:47:AM

I love the analogy ... though it is one I have been talking about for a long time (<a href="blog.magenic.com/.../On-Technical-Debt-_2D00_-Revisiting-the-Technical-Mortgage.aspx">Techincal Debt vs Technical Mortgage</a>).

Any simplistic idea about the binaryness of quality needs to go away.  I am glad the discussion is starting to appear in more mainstream places.

Aaron Erickson said:

November 12, 2007 11:15:AM

Link correction - sorry about that.

blog.magenic.com/.../On-Technical-Debt-_2D00_-Revisiting-the-Technical-Mortgage.aspx

Basil Vandegriend said:

November 19, 2007 10:02:PM

Great article, Steve. I always liked the concept of technical debt, but never thought about it to the depth you went in this article. I agree with the first comment that descriptive labels are much better than numbers, and like the labels with the short examples you proposed in your replying comment.

One of the biggest issues I have with accumulating technical debt is the risk that management will not appreciate the hidden costs associated with that debt, if not be oblivious to the existence or possibility of technical debt in the first place. I liked the ideas you suggested of tracking debt as either defects or as part of the product work list (ala Scrum) in order to make technical debt more visible.

Post a Comment:

 
 

Steve McConnell

Steve McConnell is CEO and Chief Software Engineer at Construx Software where he consults to a broad range of industries, teaches seminars, and oversees Construx’s software development practices. In 1998, readers of Software Development magazine named Steve one of the three most influential people in the software industry along with Bill Gates and Linus Torvalds.

Steve is the author of Software Estimation: Demystifying the Black Art (2006), Code Complete (1993, 2004), Rapid Development (1996), Software Project Survival Guide (1998), and Professional Software Development (2004). His books twice won Software Development magazine's Jolt Excellence award for outstanding software development book of the year.

Steve has served as Editor in Chief of IEEE Software magazine, on the Panel of Experts of the SWEBOK project, and as Chair of the IEEE Computer Society’s Professional Practices Committee.

Steve received a Bachelor’s degree from Whitman College, graduating Magna Cum Laud, Phi Beta Kappa, and earned a Master’s degree in software engineering from Seattle University.
Contact Steve