There is no Technical Debt

There is no “technical debt”, there are only badly-implemented features.

Perhaps the situation isn’t that bad. Even so, each time I see the phrase technical debt used in a discussion, I fear something bad is about to happen – or be justified. Somehow, a fairly dicey decision is about to be turned into a solid, objective decision by means of applying terminology.

Compare and contrast:

1. “The team will need to implement this in an unscalable and badly tested fashion in order to meet that deadline.”
2. “The team will need to take on some technical debt to meet that deadline.”

The first correctly identifies that risks are being taken on that may cause problems later. The second papers over this with some jargon which implies an objective evaluation has taken place over a well-understood problem space. Likely, nothing of the sort has happened.

Why has this initially appealing metaphor become misused? I think that it’s because the metaphor fails to live up to its promise of being useful in quantifying the risk caused by taking technical shortcuts. This is compounded with the fact that the costs of technical debt become invisible and so tend to increase over time rather than being “paid back”.

I think the name “technical debt” contributes to both of these problems. Firstly by conflating technical shortcuts with financial debt and secondly by reducing the problem to a technical rather than product level issue.

A failed metaphor

The core utility of the metaphor is in capturing the idea that cutting corners now may be beneficial in the short term (e.g., releasing a feature earlier), but will cause higher costs overall than not cutting those corners (e.g., the feature may be unreliable or the product is harder to change). Technical debt thereby provides a useful metaphor for less technical people in understanding that a hack-job is being carried out in order to meet a target.

However, the similarities end there. Using the term “debt” implies a rational process can be used to decide when to take on technical debt because of the term’s obvious connection with financial debt. Financial debt has a long history and rich suite of tools available to evaluate the risk associated with taking it on. Unfortunately, technical debt shares few features with financial debt. Worse, none of the shared features are those which allow debt to be effectively managed and put to productive use.

The main shared feature is the one described above. Taking on the debt incurs greater costs in the future than the current value of the debt. In finance, bringing money forward in time costs money.

Financial debt is a useful tool because it can be easily reasoned about. Take a loan. One knows the immediate pay off: the amount borrowed. One knows the future cost: the interest combined with the payment period. Combining these with various predictions about the future in a model can give a reasonable idea of whether the expected return for taking on the debt is greater than the costs associated with this. As predictions are uncertain, even this only gives a balance of probabilities rather than certainty.

Contrast this with technical debt:

• The immediate pay-off is unknown. Estimations for work are inaccurate; often to a large margin. This means that, even if both options are given estimates, the error margin on each estimate probably makes comparing the estimates almost meaningless. So one doesn’t know the savings from taking on the debt. The feature may not be ready on time anyway.
• The cost (interest rate) of servicing the debt is also unknown.
  • It may cost anything from nothing (the feature is isolated and little used) to everything (the service becomes unreliable, all customers leave).
  • Fixing a problem down the road is unknown work. Any estimates made now are invalid as the system will have changed in unpredictable ways by the time the fix is made.
• The payment period is unknown.
  • No one knows when the debt will become so problematic it must be repaid.

This means that in taking on technical debt, there is little way to work out whether the debt is worth taking on. You don’t know whether it’ll help much hitting a ship date, you don’t know what problems will be caused down the road and you don’t know how long your codebase will be afflicted with the problem. Your cost/benefit ratio is ?????? : <possibly release a feature with revenue which might be X>.

I mentioned earlier in working out whether to take on financial debt, one has two knowns and one unknown. By contrast technical debt has unknowns for all three. Therefore, a team taking on technical debt cannot use many of the tools used to successfully assess and manage financial debt. The only certainty is that of taking a technical route you’d rather not take.

In addition, it can be hard to see the cost even when it is being paid: when slower development caused by technical debt becomes the norm, it’s hard to recognise the soaring costs of servicing the debt in terms of the opportunity costs inherent in a less nimble product which is less able to respond to changes in the market. That the opportunity cost is hidden obscures the size of the payback in fixing the problem. The only cost that is seen is that of the time spent to fix the debt, which underestimates the real (and increasing) cost substantially.

From this it is obvious the metaphor is of little use beyond the superficial; therefore I’d consider it failed, or at least used far beyond its useful arena.

An invisible problem

Even if we found ways to quantify technical debt, a further problem is that it tends to become invisible and so languish unpaid. In part this is to do with the total cost being vastly underestimated. However, this is also caused by the way organisations handle it.

Taking on technical debt is usually a compromise between the technical team and the product team. However, paying it back is the sole responsibility of the technical team – resulting in the dreaded, unspecific “paying back technical debt” item in an iteration plan. Placing responsibility for resolving technical debt solely with the technical team and removing it from the product team means the product team lack the understanding to make future good decisions about whether to take on a debt; taking on the debt feels free if you don’t have to deal with the consequences.

I think part of the cause of this is framing the decision in technical terms, that is, as “technical” debt. Most of the cost of technical debt is borne by the product (fewer features, less reliable, harder to change); perhaps it should be “product debt”.

I guess it’s called technical debt because the units are technical (e.g., using a difficult to reason about class structure which causes bugs) rather than product (e.g., a missing feature). Still, the code is the product even if the product and technical teams don’t see it that way, and problems in the code will result in problems with the product.

At any rate, framing something as a purely technical problem usually results in a low prioritisation because the problem becomes invisible to many or most of those making decisions. At some point, the problem will rear its ugly head in the form of customers being harmed and complaining – at which time it will become urgent, and, ironically, more debt will be taken on to service the existing debt.

In this way, technical debt begets more technical debt by virtue of it not being addressed; its very name causes it to be ignored.

Addressing the problem

A possible way to improve the situation is for technical teams to list the items of technical debt regularly with the rest of the organisation. Repetition causes things to be seen as more important, and so repeating the problems may result in solving them being prioritised. As technical debt causes unknown and often invisible problems (like fewer features in a product), it’s hard for the technical team to avoid being seen as naysayers if they take this strategy.

Therefore both sides need to work together to address technical debt. It should be a first-class item in the product prioritisation process, and if a decision is made not to address it, it should be removed from the product backlog, much like a bug or feature that is deemed unimportant is. Making this an active, visible decision should result in better choices being made and, when failures occur, the whole organisation can take responsibility rather than blame being lumped purely on the technical team.

As to the problem of costs being unknown and perhaps impossible to see even when paying them, perhaps more eyes on the problem will produce better solutions – visualising the counter-factual scenario of a product that’s able to quickly respond to market changes is always going to be a hard one, and one that’s unlikely to be popular when an organisation has driven itself into a mire of technical debt that it cannot hope to climb out from. If more people feel responsible, more may be done to avoid the fall in the first place.