A common question is whether bugs belong on the product backlog. Before I address that question, let me clarify that the question refers to bugs that are unrelated to functionality being coded during the sprint. If someone finds a bug during a sprint that is related to the features being worked on, the best thing to do is yell, “Hey, Mike, the boojum code is broken when I…” The second best thing to do is to stick a new task card on the task board saying “Fix the boojum code; it breaks when…”
But what about a bug found today in something the team wrote a month or a year ago? And what about the existing bug database that a team new to agile is almost sure to bring with them?
The ideal situation is to put the bugs right onto the product backlog. To see why, consider the situation from a user’s perspective: Do you think a user cares whether something is considered a New Feature or a Bug? No. The user just wants the system to behave in some way different from how it behaves today. They don’t care what it’s called.
But notice I called this the ideal situation. It isn’t always practical–and sometimes for reasons out of the team’s control or influence. Bug databases have a way of becoming embedded into organizations. The tech support group uses them as a primary way of communicating with the developers. The marketing group uses the bug database in a similar way. This makes it quit the bug database cold turkey.
In these cases, the most common solution teams use is to have two backlogs
- a product backlog of features
- a bug backlog
This approach is a bit harder on the team and the product owner but allows an agile team to work more easily with existing processes in the organization.
Sometimes when we make such accommodations to the overall organization, the accommodation can damage or destroy the agile adoption. Allowing everyone to work on five concurrent projects is an example of a crippling accommodation. Accommodating the organization’s use of a bug database is not crippling. It’s just a bit more work.
The product owner takes on most of the additional work by having to prioritize two separate work queues and then present them to the team in a more or less consolidated manner (“My top priorities are the first three items on the product backlog, then bugs #12403 and 12415, then these next two items on the product backlog…”). This isn’t too onerous as the items would need to be prioritized even if all were in one backlog.
So: Ideally bugs belong on the product backlog just like any feature request. But, that would often necessitate a significant change for the rest of the organization so two backlogs are used.
Tags: product backlog, user stories
Hi mike. We have been experimenting with both approaches. Here’s my learnings:
1) On a separate list: this easily creates sub-optimal processes, where the team will have difficulties understanding what is most important. It will also easily end up being a different process that runs through the bug backlog than through the product backlog (eg. team estimates vs. individual estimates. It is really difficult to prevent this, and therefore we turned away from having separate lists.
2) On the same list: you easily end up spending too much of your valuable time doing first point estimation and then later on hour estimation on bugs that in reality requires perhaps 1 hour to fix.
The process that we defined for bugs therefore is a compromise which I strongly feel for: we calibrated 3 types of point estimates for bugs (e.g. small= 0,5p, medium=2p, large=5p). These calibrations are based on teams experience, velocity and learnings throughout projects. On product backlog level they now very quickly identifies the bugs in terms of small, medium and large, which then triggers the points to be given. Of course if the team really know that a particular bug is huge they can give an appropriate estimate e.g. 13p.
One has to remember that the nature of a bug is different than the nature of a feature (perhaps not that much when working on legacy systems, but thats another discussion). For a bug you rarely know the solution up-front, because you’ll have to find the bug first, for a feature you can quickly form a picture of a solution up-front…
Mike,
I agree 100% that you should maintain a single backlog, and at the same time that it is difficult to get rid of the bug database. But I would say that maintaining two backlogs is not a good idea. Things can get out of hand if support team uses another tool to track support issues and sales team uses a CRM tool to track sales requests and so on. We could easily have three or four backlogs split all over the organization.
I feel that the organization must make some effort into getting the data into a single backlog, perhaps by writing scripts to automatically export data into a master list.
I expanded on my thoughts in this blog post – http://bit.ly/V9KTL
Hi,
as I am getting familiar with the concepts of agile planning, I had exactly this kind of doubt. Reading articles and books on the topic always seem to concentrate on *new* features, as opposed to defects of a (partial) product that has already been released in some form or another (internal release vs public release).
While I see your point on the ideal situation (especially from a product owner / customer point of view), keeping the old trusted defect database (which is highly visible throughout the organization) seems to be a good choice: I am afraid some of the stakeholders (senior managers) may become very nervous about the healthy of the product if they cannot run their daily queries or receive daily reports on the defects backlog, which a dedicated bug tracking system is very good at.
Thank you for sharing this.
Stefano
Stefano
I believe bugs should be on the *top* of the backlog. The reason simply is that technical debt kills velocity. Poor quality has a negative compounding effect when left unchecked. Has there been any empirical studies on agile teams who strive for a zero defect policy? BTW, I am a product owner, so yes I understand the business value trade-off when in the heat of the moment.
BTW, you can keep your old trusted database. We use clear quality for defect management and a utility pulls defects into VersionOne to be prioritized against the larger backlog.
Hi Mike
Great post, I would add to this that balance between bug fixes and new features is often overlooked and critical to health of a software product. Too much of one or the other can be deadly.
Luc
Hi Mike,
I take a little different approach. I think there are two kinds of bugs. The first is just a mistake. Someone used the wrong variable, used a wrong value, called the wrong method. Stuff happens. This type is blocking functionality that was assumed to be working. Hopefully a modern team using TDD will find these before the end of the sprint. If not, there are few things of higher priority. Track it if you have to, but get it on the next sprint backlog.
The other kind is when the team did not understand ( or was not given ) the full scope of the requirement. This one often includes interaction with other systems. This type is much more common and the priority cannot be determined by the team. The Product Owner team must make sure that they understand the value of the missing or wrong functionality and present that story back to the team for sizing and prioritization.
In either case, I think that the work needs to be prioritized alongside the other stories. What is the business (customer) value? How much effort and risk is there in completing this story?
Scrum is about delivering business value quickly and repeatably. Prioritization of business value must drive the project. Prioritize all work together.
Rod Claar
I think the most important question is really how would the product owner prioritize it? Let’s say the current sprint is targeting a specific module in a system and the release is looking like it will be in a couple of days. All of a sudden here comes a bug from a diff module that must be fixed “ASAP”, what do you do? It seems to me that whether you place in the current backlog or a separate bug backlog doesn’t matter as much as how you are going to shift around resources and plans to accommodate this bug, don’t you think? I’d be interested in hearing proposed solutions to this hypothetical.
A lot of replies came in so I’ll try to reply to common themes among them:
Thanks, everyone, for sharing your comments.
Interesting post Mike. I would agree that this presents an issue, but I guess I think the real issue here is not which backlog they go on. Regardless of where you put them, the prioritization problem is still there. What takes precedence? Bugs so you can prevent technical debt as Paul says in his comment above? or features so you can get new stuff out to market as quickly as possible? We’re debating this right now on a relatively new product we’ve taken to market. We haven’t fully figured it out yet, but we’re headed in a direction of controlled technical debt rather than no technical debt. I’m of the belief that pursuing “no technical debt” is too much purist, not enough pragmatist. So, our challenge is going to be deciding just how many bugs and what kind of bugs are acceptable. We would then set a threshhold and keep the issues/debt from growing. This seems to be the only way for us to quickly deliver new features to market without making our lives increasingly more difficult by also increasing debt.
Thanks,
Matt
Hi Matt–
Technical debt is often misunderstood. The coin was termed by Ward Cunningham, who speaks about it on YouTube. His one or two page paper on it is also online.
We don’t need a goal of having no technical debt, any more than we need a goal of having no personal debt. Used correctly, debt can be a helpful thing. If allowed to build up, though, it can be crushing. So, I think you’re on the right path with pursuing a controlled amount of technical debt.
“Fix This Defect” Is Not A Feature…
…There is an important property of defects that make scheduling their fixes a bit more complicated, though: a defect in the software system also denotes a defect in our process – a hole that allowed the defect to slip through. And not fixing that pro…
This is definitely a very common question for the Scrum Teams and this post clarifies it a lot.
I have been maintaining bugs as part of the Product backlog but the the question that always bothers me is should we include the estimates into the Team velocity. I think most of the times when the bugs were introduced by the team due to poor design/ code/ testing why should the team get that into the velocity because in reality there was no new business value added to the product as this should have been actually delivered during original feature development. Thoughts?
We use only one prioritized backlog to track both bugs and features. I fully agree with Abhilash just above this comment. Bugs should not contribute to velocity, as they do not add business value.
Great post Mike! I have found that keeping the Product Backlog and the Bug Backlog as separate lists very useful. This allows us to consider defect rates and other such metrics not associated with features. The approach then becomes. If you find a bug and can fix it, do so, if you can’t log it into the Bug Backlog. During ever Sprint Planning meeting the Bug Backlog is reviewed and bugs are put on the Sprint Backlog and prioritized alongside features. I have also found that during the bug list review we find items “that have been fixed” we put these items on the Sprint Backlog in the “Ready for Test” state so QA can verify that they have indeed been fixed.
Hi Chad-
Thanks for your comment. I’m happy to hear others using a similar approach. I know this isn’t ideal (we’d love to have no bugs, even on the legacy project we move to agile) but it’s a fact of life. I really, really encourage companies to work to eventually get rid of the bug backlog because doing so is quite achievable for the development team. But, if this causes other organizational strife, it’s just not a battle worth fighting very hard for in most companies as there are so many other problems that will have bigger paybacks.
I find tracking bugs to be waste. If you have a bug backlog I suggest deleting it and getting rid of any bug tracking software. Any bugs reported needs to be fixed as soon as reported.
Tracking bugs is like keeping a diary of bad memories and have them thrown at you every morning when you wake up. It’s really demotivating for the team.
If you are in a situation where bugs needs to be tracked because there is no time to fix them right away, you’re in a fight that you’re bound to loose. By doing that exact action you’ve stated “I’ve given up”.
The zero bugs goal is possible. The software industry should start taking it seriously and our customers should stop accepting software with bugs.
I’d like you opinion on a tangent problem we are dealing with, in our agile adoption and defect management.
To set the stage, my org is in the early phase of adopting scrum. We have about 300 people and 20-30 million lines of code. We sell, develop and maintain an enterprise level suite of heavily integrated applications to external customers (meaning we are not IT). These very technical products have suffered for many years (as much at 20 + years) of inconsistent development and QA practices and as a result, are very buggy. So we are dealing with sprints, that in many cases are (almost)all defects. Granted the defects may be poorly classified as such and could be more of a feature change. We don’t specifically have an issue with prioritization of stories versus defects, our PO org is managing that part fine. We treat these pre-existing defects just like stories, they are in the backlog, they have story points, and when committed to a sprint, get a task breakdown.
Here is the issue – when it comes to a demo meeting at the end of the sprint, the teams seem to feel that demoing bug fixes is too time consuming and wasted effort. They might have fixed 30-40 defects in a 2 week sprint, and setting up a unique base state environment to demo each one (our software runs on several OS’s) takes more than the 2 hour demo time-box. The PO, stakeholders etc start to fall asleep and don’t engage in the meeting. The usual comment from the PO is that, if the team fixed a crash, what value does it have to show the was-is in the demo meeting? The PO would rather take QA’s word for it and move on. So unless that sprint team was “lucky enough” to also have a feature story in the same sprint, they now try to cancel the demo, stating there was nothing of interest to demo. I think we should stick to our guns and demo everything. If the defect had no business value, then the PO would not have prioritized it as they did. Any suggestions?
I think you’ll find the answer in remembering that we call the meeting the “sprint review,” not the “sprint demo.” The purpose of the meeting is to get feedback on what the team has done so that this feedback can inform the planning of the next (and subsequent) sprints.
When we remember to call this a sprint *review*, I think it becomes more clear that the team does not need to demonstrate every bug fix. I’d far prefer to hear about the bugfixes that were taken care of. Perhaps, “And we fixed 38 defects. Here’s a pie chart showing how they split by severity. Here’s a list of customers who were affected by severity 1 bugs that are now fixed. Any questions? Any comments on which bugs or which types of bugs we should fix next. I know our PO is debating whether to fix all priority 2 bugs anywhere in the system or to fix all bugs in the account maintenance area. Any opinions on that?”
Keep the purpose of getting useful feedback or guidance in mind and the meeting will run more smoothly, quickly and with less pain for all attending.
In regards to what Paul Relf said, I don’t agree at all that bugs should automatically be put onto the top of the product backlog. This means they should be worked on before other items in the backlog. Why?
Secondly, I wanted to point out my views regarding the handling of bugs. I consider the sprinting team to be like a big old machine, with input, and output. The output is a shippable product, and the input is little pieces of paper with things written on them.
The product owner’s job is to run around between users and stakeholders, gathering information to be written on the bits of paper, then to write these papers, order them, and then to feed them, in order, into the machine.
The machine should output the resulting work in the same order as it was input by the product owner.
I guess that what I’m saying is, the only person who needs to use terms such as ‘bug’ and ‘feature’ – the only person who needs to think about these things, is the product owner.
Neither the stakeholder, nor the developer, nor the tester, nor Mrs. Enduser care at all about what you call the work. Similarly, they don’t care about you and your backlists. So don’t ask them to help you decide whether or not to use a single or multiple backlists, or what to call the work your team does.
Hi Jacob–
Very well said and a nice analogy. This fits exactly with what I was trying to say when I said that having a second work queue should cause additional work only for the product owner. It should be mostly a non-issue to others.
Thank you very much for helping clarify that point and doing so better than I did.
I would disagree with Jacob to the extent that if just PO decides what gets prioritized only based on BV without considering inputs from the team based on technical/architectural implications, we run the risk of sub-optimizing BV. I like Matt’s “controlled debt” idea. The difficulty with that is how do you know what that level is. Also how do you maintain that level? This sounds like a good concept on paper, but very difficult to implement in practice without running the risk of degenerating the system.
So, I prefer to prioritize known bugs over new features. If anything, this would allow us to achieve that “controlled debt.” Because there are always bugs in any system, it is just that we don’t know. So, fixing the knows bugs is always important.
It might be difficult for a legacy system with loads of bugs already on the backlog. For new systems, if bugs are always removed before new features are added, it never becomes an issue even from business perspective, because it should be 10-20% of your sprint backlog. Even that would be high and would recommend to review the engineering practices. If you start falling behind on fixing bugs, then it becomes difficult to catchup and maintain a controlled level of debt.
Hi Mike,
I agreed that sometimes we need to have separate backlog for bugs. In my organization, we do have different bug tracking systems. The Agile teams need to follow this kind of policy also. Although it is not perfect agile practice, but it works well. Each time, during the planning meeting, PO and team will make the sprink backlog together to get items that should be completed from both user story backlog and bug system. And also some teams will consider to leave 1~2 days in each sprint for bug fixing.
We always encourage team to fix all of the bugs that found in one sprint. But it is acceptable that some defects cannot be fixed because of dependency, legacy issue or something else.
The problem we have is about how to estimate the bug. Now we are trying to estimate the effort (hours). But I am wandering if it is possible to estimate bug with points. Since points is used to indicate the relative size, it seems OK to use it for bug estimation. I think it will be better to setup a unified estimation method for all of sprint backlog items. The team’s velocity can be figured out easier.
I have similar question as Dingshan Li.
Few of you’ve mentioned that defects should not change velocity as defects do not brings business value to product.
Does it mean there are no estimation of effort? Complexity? Are you just tracking remainig time vs. estimated duration?
What in case when our sprint is to manage defects found while user acceptance testing?
You know, sometimes you have to adapt your Scrum process to customer’s waterfall process. Typicaly if customer is thinking about new version deployment as internal project – banks, etc.
A single queue is a must for me, otherwise prioritising for the project owner becomes too complex. Having a dedicated subset of developers that call themselves maintenance programmers, who typically don’t get involved in sprints, is the way my last organisation found to handle the bug workload. Some people simply enjoy it and are therefore good at reading others code and making it better.