Snake Pit... Taking the Poison out of Technical Debt

Projects build in technical debt and maintenance groups remove it—if your organization has a maintenance group. Technical debt accrues in any product, whether or not it has a technical component. It is the result of taking shortcuts when building the product. Sometimes it is the result of not having enough time, on other occasions it is due to not having the right tools. Anything from the implementation of the software component to light fixture can have technical debt. Promises are made to correct it later, but later never comes.

Case Study: The "Snake Pit"

Recently there has been a resurgence in the popularity of the article Using Agile in a Project Recovery. As with any new implementation, there were a few problems—technical debt was one. One was experienced in the third iteration; a bug was introduced in the release. Because of the short development and testing cycle regression testing was constrained to a logical set of tests based on what was changed. However, this bug was in an area that was not worked on, so regression testing missed it. I asked the architect and the QA lead what went wrong. The architect regrettably shook his head and relied, "The Snake Pit." I had heard the developers refer to this before. It was a section of code that had been modified too hastily by too many people, too many times. The code was a mess, poorly written, and not well understood. I asked the architect to come back and give me a recommendation.

The next day Bob came back and told me that the four-week release cycle could not continue. The code was written so poorly that we would continue to trip over unexpected functionality. We got the developers together and talked about the options. The developers were very frustrated. In my haste to implement the process, I had missed telling them the importance of refactoring the code base. I told them that their estimates had to include a sensible amount of time to refactor the code.

Two guidelines were defined to help:

Changes had to be sound. They could not be hurried; they needed to be able to be completed in a short amount of time.
Additionally, each iteration had a few days set aside for unanticipated refactoring. When they touched code that was poor quality, they were supposed to change it.

The developers were ecstatic. They had never been given this leniency to use their discretion at how to do their work. Needless to say, the organization was not one to trust the judgment of its employees. This, and the implied trust that went with it, made pride soar.

Technical Debt's Effect

Trying to fix technical debt in a product can be a monumental, especially if it has accrued over time. Estimates for project work are often created with the assumption that the code base is sound. However, when the team starts working, they find that the product was not built to best practices or shortcuts were taken and now, before the enhancements can be added, the mess needs to be straightened up. This effort can be significant. The customer, of course, does not want to hear any of this, since they do not feel they caused it. It is up to the project to fix the code and make it maintainable.

It is worse than this, though. Technical debt is not just from poor practices. Products simply go out of date. Techniques and tools change, new products emerge, and better methods make the old structure passé and fragile. When adding new functionality, it usually does not makes sense, and sometimes it is impossible, to continue to use the old style. There are times that a remodel, or refactor, is a necessity. No one is to blame, it just needs to be added to the scope.

The Maintenance Group

Attending to technical debt is a never-ending process. A maintenance group is the best group of people to do this work. They can do this between projects providing a solid code base for the next project. However, budget cuts have relegated this to a luxury. This is a short path to a non-maintainable, costly, bug laden product.

Handling Technical Debt in a Project

As highlighted in the case study (see inset), technical debt is a serious issue. The best defense is a good plan. Discovering the problem early is critical since it gives the team time to understand its breadth and potentially start working on fixing it prior to working on the project's product.

Delicious

The worse case is when the project team tries to remove technical debt and the project is not equipped to do it. This can be a result of two conditions—the project does not have the funds or management has made a decision to make a quick fix. Technologists hate these situations since they feel they are developing substandard product and will need to fix it at a later date. The case study in the article Finding Religion, Trusting Management describes just such a situation. Project managers need to pay special attention to stop this work.

Tags: IT Failure, Project Management, Scope Creep, Technical Debt