Technical Debt Decision Making

  1. Posted on December 12, 2007 11:18:AM by Steve McConnell to 10x Software Development
  2. Technique, Technical Debt, Design, Maintenance

[This is an expansion of one of my comments on an earlier Technical Debt post]

When you get to a point where you are debating taking on technical debt, people normally consider two possible paths, one of which is the "good but expensive" path and the other of which is the "quick and dirty" path. When teams reach that decision point, they often estimate the good path and the quick path. Those estimates will help inform which path the team should choose at that moment. But there are three more issues that should considered.

The first additional issue to be considered is how much it will cost to backfill the good path after you've already gone down the quick path? Backfilling the good path will typically be more expensive than just following the good path in the first place because the work will include ripping out the quick code, making sure you didn't introduce any errors while doing that, then adding the good code and going through the normal test & QA processes. The "ripping out" part makes it cost more to implement the good path later than it would have cost to implement it in the first place. And of course you've already incurred the cost of the quick path, so the real cost is the sum of the quick path + the good path + the cost to rip out the quick path.

If the code is really well designed the "ripping out" cost can be minimal, but I think that's the exception.

The second additional issue that should be considered is the interest payment on the technical debt. I.e., if you choose the quick path now, how much does that slow down other work until you're able to retrofit the good path? The size of the "interest payment" depends very much on the specific case. Sometimes the "interest" is really just the cost of ripping out the quick code and of implementing the good code, which isn't really interest, per se. It's more like a late payment fee. Other times the quick and dirty approach does create ongoing interest payments by making related work in that same area take longer.

This leads us to the third issue that should be considered: Is there a path that is quicker than the good path and that won't affect the rest of the system? In other words, is there a quick path that can be isolated from the rest of the system in such a way that it doesn't create any ongoing interest payment/make other work more difficult? In my experience teams often turn the technical debt decision into a simplistic "two option" decision -- good path vs. quick and dirty path. Pushing through to a third option is important because often the best path is the one that is fairly quick, albeit not as quick as the pure quick and dirty path, and whose adverse affects are better contained than those of the pure quick and dirty path.

With those three options, the decision table for deciding which kind of technical debt to take on could look something like this (assuming a labor cost of $600/staff day):

Option 1: Good Path

Immediate cost of Good Solution: 10 staff days
Deferred cost to retrofit Good Solution: 0 staff days

Option 1 cost now: $6,000
Option 1 cost later: $0
Option 1 lifetime cost: $6,000

Option 2: Pure Quick & Dirty Path

Immediate cost of Quick & Dirty solution with possible interest payment: 2 staff days
Deferred cost to retrofit Good Solution: 12 staff days
Estimated cost of "interest payments": 0.5 staff days/month

Option 2 cost now: $1,200
Option 2 ongoing cost (interest): $600-$1,800 (assuming good solution is implemented within 6 months)
Option 2 cost later: $7,200
Option 2 lifetime cost: $9,000-$10,200

Option 3: Quick but not Dirty path

Immediate cost of Quick & Dirty solution with no interest payment: 3 staff days
Deferred cost to retrofit Good Solution: 12 staff days

Option 3 cost now: $1,800
Option 3 ongoing cost (interest): $0
Option 3 cost later: $7,200
Option 3 lifetime cost: $9,000

In this example, either Option 2 or Option 3 is an attractive short-term alternative to Option 1. That is, either $1200 or $1800 is a fraction of the cost/effort of $6000. But if you select Option 2 you saddle yourself with an obligation to revise the code later--either you reimplement the good solution, which costs more, or you keep paying interest, which costs more. When you select Option 3 you introduce the possibility of choosing never to pay off the technical debt, because there isn't any ongoing penalty, and so there isn't any urgency to pay off the debt.

Bottom line: When facing the prospect of taking on technical debt, be sure to generate more than two design options. Don't oversimplify technical debt decision making to just the two extremes.

Hisham Abboud said:

December 12, 2007 1:56:PM

Great post, as was the original one as well.

I recently wrote about application development vs. consulting.  I couldn't help but wonder if application developers are more likely to use the good path because they *know* that doing otherwise will come back to haunt them, whereas in-house developers and consultants are more inclined to use the quick path.

Ryan Smith said:

December 12, 2007 2:18:PM

Steve,

These technical debt posts have been great, and I love the way they put the cost of coding decisions in perspective for someone with a financial background rather than a technical one.

I look forward to your next post.

Ryan.

Ngu Soon *** said:

December 12, 2007 6:15:PM

Good post.

But software estimation is notoriously hard, regardless of whether you are estimating the time or the monetary cost. How do you generate the money value associated to the options? I think different people will have different idea on how much the options are worth. And that's where the controversies lie

Steve McConnell said:

December 12, 2007 6:37:PM

> But software estimation is notoriously hard ...

In order to justify taking the quick and dirty path *at all*, you have to have some kind of estimate that says the quick and dirty path is less effort than the good path. So if you're thinking of taking on technical debt, you're already doing some kind of estimating. The costs are simply the number of effort days times $600/day.

Steve Smith said:

December 12, 2007 6:48:PM

Steve, I like and agree with your analysis. However, it doesn't include a vital element -- how the decision is made.

Let's say the decision is ultimately decided by Ellsworth, the pointy haired boss. Ellsworth hasn't told you and the other members of the development team he will receive a bonus of $500 if the product is delivered in 3 days. And based on his vast experience, which isn't much, he believes the need for a rewrite is roughly 50%.

Ellsworth tells the team that the company needs the value the product will deliver now. He chooses the Q&D option. He based his decision on a model of the world that was likely different then the development team and, more importantly, invisible to the development team.

Your kind of analysis will work wonders in organizations where decision makers put their cards face up on the table and make a rational business decision. For those that don't, choices will be made that, on the surface, don't make sense to anyone but the decision maker.

I believe our job is to put the economic analysis on the table and to influence the decision maker to put his or her cards face up on the table so whatever decision is made people learn from it.

I know it's a Utopian dream, but it's what I believe in.

Happy Holidays,

-Steve

Bogdan said:

December 13, 2007 5:24:AM

Unfortunately, the Q&D path is taken very often (along with the technical debt) on software projects that demand delivery "now".

One factor is that as we all know, development estimation is not really an exact science and some developers tend to decide that in order to fulfill their goal the only solution is quick and most of the time "dirty".

Bogdan

Neil Galloway said:

December 13, 2007 10:50:AM

Good points.  

One item I am always trying to think of it the importance of the change and "will it ever be revisited for a retrofit?" or in a small rapidly changing business situation you could also ask "we need this now, but requirements and process will be changing in the near future, so the retrofit will have a high probability of being necessary anyways".  

I am stretching it a bit, but my point is that your arguments are excellent for well defined, mature requirements type of situations.  They won't work as well otherwise.

Jim Fiorato said:

December 15, 2007 8:49:PM

Yet another good article.

I think to Neil's argument, Steve's point about the "interest payment" is what's important, whether or not you've got a "mature requirements type of situation".  If you can find a simple way of tracking your interest payments (excel, MS project, issue tracking system, Quick Books :P), you can start creating artifacts that quantify the technical debt you have.  You can do this despite how mature your process is (or you can think of it as a step towards being more mature).

If you have a running tally of what your interest payments are, and where they are being paid to, then during planning you have some ammunition to bring to the gold owners.  You can say "hey, we're paying $1300/month for this Q&D code, do you still want to be paying this?".  If the answer is no, re-estimate the good solution (chances are things have changed), and provide the alternative.

In most cases the intent to rip out the old Q&D solution gets lost in a sea of other intended retrofits.  None of which can be easily prioritized over the other.  The debt just grows and grows, and architects and developers can become paralyzed figuring what they should fix first.

Lionel said:

December 16, 2007 12:26:AM

I don't really see the point of technical debt as a concept. it seems quite limited.

your 3 examples work only if the coders have a perfect description of what to do.

Most of the time, the requirement are a moving target. So the team will have to rewrite the code anyway, because of the refined requirement (or better understanding of the underlying problem). Technical debt is simply inevitable.

Most important is to know how to manage this inevitable technical debt. And XP/Scrum seems reasonable method there.

(oops didn't see Neil Galloway's comment)

kcmarshall said:

December 18, 2007 11:28:AM

Unless some requirements were set aside or deprioritized, I don't see a distinction between Options 1 & 3.  

If requirements are set aside, then perhaps they aren't _requirements_ but merely goals or nice-to-haves.

If requirements are deprioritized or delayed, this is a change in project scope.  This would be a welcome and healthy compromise by project management between priorities (schedule vs. features).

The ability to predict no ongoing "interest" suggests a degree of design which is likely to match the Good Path.  This suggests that reducing design time won't produce 70% reduction in development time.

Without testing/QA, I don't think you can have confidence that requirements have been met and that "interest" can be avoided.

Where, then, will the time savings be realized to cut 70% of development time?  Ultimately, I may not understand what Quick & Dirty means to you.  When my projects go Q&D, I anticipate that design and quality will be compromised.

Thanks!

Kevin

Steve McConnell said:

December 18, 2007 12:47:PM

Kevin, You might check out some of the examples of technical debt from my first post on the topic. Here's another example that illustrates the answers to many of your questions.

Let's say that you're developing a large app that requires, among other things, 5 reports. Your plan is to design a set of custom report writer classes that are specific to the nature of the application you're working on and that will make generation of additional reports later on easier. For the sake of expediency, you're being pressured to write the reports using your database's built-in report generator instead. That makes writing the initial set of reports easier (mostly), but once that initial "starter kit" of reports are done, later reports will be more difficult.

Here's how the options I listed before might play out in this example:

Option 1: Good Path

Write all the custom classes, test them, and implement the 5 reports using those classes.

Immediate cost of Good Solution: 10 staff days

Deferred cost to retrofit Good Solution: 0 staff days

Option 1 cost now: $6,000

Option 1 cost later: $0

Option 1 lifetime cost: $6,000

Option 2: Pure Quick & Dirty Path

Use your database's built-in report-writer. This makes generation of 4 of the 5 reports very fast. The fifth report is harder than it otherwise would have been. Going forward, you expect that each report after the first 5 will be harder to write than it otherwise would have been. Moreover, if you ever do implement the originally planned custom report writer, you'll have to go back and regenerate each report in your custom report writer.

Immediate cost of Quick & Dirty solution with possible interest payment: 2 staff days

Deferred cost to retrofit Good Solution: 12 staff days

Estimated cost of "interest payments": 0.5 staff days per report beyond the first 5 (because later reports written using the database's report writer will be harder to write than they otherwise would have been)

Option 2 cost now: $1,200

Option 2 ongoing cost (interest): $300/additional report

Option 2 cost later: $7,200

Option 2 lifetime cost: $9,000 + $300/additional report

Option 3: Quick but not Dirty path

Use your database's built-in report writer, BUT you wrap the database code in a translation layer that presents the same interface that the planned custom report writing code would present. Writing the translation layer takes two extra days, BUT the retrofit cost will be lower because the code for each report will not need to be changed when you swap out the database's report writer and swap in your own custom code.

Immediate cost of Quick & Dirty solution with no interest payment: 4 staff days

Deferred cost to retrofit Good Solution: 9 staff days (1 day less than Option 1 because the interface-creation work is done during the Q&D phase)

Estimated cost of "interest payments": Zero (because later reports can be written to the custom-report interface)

Option 3 cost now: $2,400

Option 3 ongoing cost (interest): $0

Option 3 cost later: $5,400

Option 3 lifetime cost: $7,800

In the short term, Option 3 costs more than Option 2 ($2400 vs. $1200). In the long term, it costs less ($7,800 vs. $9,000+$300/report). But most significantly with Option 3 you don't incur any ongoing interest payment that forces you to go back and implement the originally-planned custom code; you can delay that decision indefinitely.

Like I said in the post, I think it's useful to generate several different design options, rather than just the "quickest and dirtiest" on the one hand vs. the "most pure" on the other hand. Often a hybrid approach ends up being the best option.

Alex said:

January 8, 2008 1:49:PM

Steve, great article as always.

The problem we are facing now is that we have accumulated a large amount of Technical Debt with our product. We'd like to be able to measure and quantify this debt so that we can steadily work towards reducing it.

Do you have any advice you can give on the best way of measuring existing technical debt that has accumulated over many years?

talmans said:

January 8, 2008 6:22:PM

This is an excellent topic.

This is a big issue in Software As A Service and hosted solutions.   With all the acquisitions and mergers it becomes important there too.  

This question is central to Maintenance programming which went out of vogue many years ago and has come back with the rise of SAS solutions.

Good programmers and good maintenance programmers can navigate the quick but not dirty solutions and keep the codebase tracking towards a reference architecture that offers profitable low cost solution.

I lived this for many years and I agree with the quick but not dirty approach. I know that this works but it can be challenge to ensure all options are considered.  It's tough to not get cynical here.

Often the decision is made before viable option 3's are considered. :)  

Great article.

talmans said:

January 8, 2008 6:30:PM

To Alex's question:

We had success defiining "debt" projects and estimating their effort.  These would get prioritized along with other work.

We had two major challenges with this.  

1. Inlcuding these projects as part of planned feature changes so you can kill 2 birds with one stone or 1/1/2 stones.

2. Prioritizing these infrastructure changes ahead of some customer feature commitments was very difficult.  Resolving debt doesn't generate revenue, instead, it lowers cost so making a persuasive business case to do the work is a challenge.

I'd be interested in ways of making this business case vs adding new features that customers are willing to pay for.  This is why I advocate option 1 above.

if anyone has experience with this please chime in.  

Post a Comment:

 
 

Steve McConnell

Steve McConnell is CEO and Chief Software Engineer at Construx Software where he consults to a broad range of industries, teaches seminars, and oversees Construx’s software development practices. In 1998, readers of Software Development magazine named Steve one of the three most influential people in the software industry along with Bill Gates and Linus Torvalds.

Steve is the author of Software Estimation: Demystifying the Black Art (2006), Code Complete (1993, 2004), Rapid Development (1996), Software Project Survival Guide (1998), and Professional Software Development (2004). His books twice won Software Development magazine's Jolt Excellence award for outstanding software development book of the year.

Steve has served as Editor in Chief of IEEE Software magazine, on the Panel of Experts of the SWEBOK project, and as Chair of the IEEE Computer Society’s Professional Practices Committee.

Steve received a Bachelor’s degree from Whitman College, graduating Magna Cum Laude, Phi Beta Kappa, and earned a Master’s degree in software engineering from Seattle University.
Contact Steve