A common question that we get from teams who are new to agile is whether you should assign points (sizes) to defects. From a Disciplined Agile point of view we know that the answer will vary depending on the context of the situation, or in other words “it depends.” The really disciplined answer though is to say on what it depends, which is what we explore in this blog.
Rod Bray, a senior agile coach at Scott Ambler + Associates, suggests this principle:
Don’t reward people for shoddy work
An implication of this is that if a team is injecting defects, they’re producing shoddy work, then it makes sense that their velocity suffers when they have to spend time fixing those defects. The decrease in velocity is a visible trigger to investigate, and address, the cause of the productivity loss. But what if the “shoddy work” wasn’t caused by your team? What if the “shoddy work” was previously acceptable to your stakeholders? Hmmm… sounds like the context of the situation counts in this case. The following flowchart works through common thinking around when to size a defect. Note that this is just our suggestion, your team could choose to do something different. Let’s work through common scenarios to understand whether defects should be sized or not. These scenarios are:
- Is it existing technical debt?
- The defect is injected and found by the team
- The defect is found by independent testing
- The defect is found after the solution is released
Do you size the repair of existing technical debt?
The quick answer is sometimes. For the sake of discussion we’ll assume that this technical debt has existed for a long time, potentially years. From a purist point of view technical debt may be injected at any time (this is obviously true) and as a result may only be seconds old. In the case of technical debt that is very new, refer to the logic below.
The key issue is whether fixing the defect is going to be a substantial effort. This of course is subjective. Is the work substantial if it’s less than an hour of effort? Several hours? Several days? Several weeks? This is a bar that your team will need to set. Something that will impact your schedule is likely to be substantial (and something your Product Owner is likely going to want to prioritize so that it can be planned for appropriately). Clearly a defect that takes several days or more to fix is substantial and one that takes less than an hour is likely not. Something that takes several hours or a day or so is likely something you need to think about.
Do you size defects injected and by found by the team?
No, the exception being technical debt (see above). This falls under the principle of not rewarding the team for shoddy work.
Do you size defects found by independent testing?
Some teams, particularly those working in regulatory environments or working in complex environments, may be supported by an independent test team. An overview of the independent testing strategy is presented below. With the exception of the defect being found in existing technical debt (see above), the defect should not be sized. Once again, the principle described earlier holds. Of course if you don’t have an independent test team supporting your team then obviously you can ignore this question.
Do you size defects found in production?
The answer is sometimes. As you can see in the high-level lifecycle diagram below, the DA toolkit explicitly recognizes that change requests are often identified by end users of an existing solution. These change requests are effectively new requirements that should be prioritized and worked on appropriately. Many organizations will choose to distinguish between two types of change requests, defects and enhancements, so that they may be treated differently. The issue is that defects are often tracked and, if the team is smart, they are analyzed to determine how they got past the team in the first place. Additionally you may find that defects and enhancement requests are charged for differently (a topic for a future blog).
If you don’t distinguish between defects and enhancement requests then there’s nothing to do, you simply treat the change request as a new requirement and size it like you normally would. But if you do distinguish between types then you need to think about it a bit. If the defect is found during a warranty period then it shouldn’t be charged for. Sometimes, particularly when work is being outsourced, there is a warranty period of several weeks or even months after something is released where the development team is expected to fix defects for free. In this case you likely wouldn’t size the work in line with the principle described earlier. Once you’re out of the warranty period then it’s likely you want to assign points to it: The functionality in question passed testing and was accepted by your stakeholders, so they in effect have taken responsibility for it.
To summarize, the answer to the question “Should we assign points to a defect?” is “it depends.” In this blog we explored in detail why this is the case and described when and when you wouldn’t want to assign points.
I’d like to thank Kristen Morton, Rod Bray, and Glen Little for their insights which they contributed to this blog.