Should a team that is estimating in story points ever re-estimate? This is a question I’m commonly asked and would like to address here.
Most people have a natural feeling that re-estimating is somehow wrong but they can’t quite say way. I’ll encourage those individuals to stick to their hunches, and hopefully I can provide of the reasoning that supports your natural inclination that most re-estimating is wrong. Philosophers talk about two types of knowledge. The first is a priori knowledge, which is knowledge before you experience something. Let’s call this knowledge-before-the-fact. This is the type of knowledge we have when we estimate something. Before I estimating development of the new search screen I think it’s about 8 story points, because it seems to be about the same total effort as some other 8 point story. The other type of knowledge is called a posteriori knowledge by the philosophers. This is knowledge after the fact.
When we estimate it is important that we not mix knowledge-before-the-fact with knowledge-after-the-fact. Suppose you are looking at a Scrum product backlog that has just been estimated with none of the work started. Each of those estimates was given before-the-fact (a priori). Now suppose you are looking at the same project a few months later. You’ve got a list of completed work, some of the items on that list still show their original, before-the-fact estimates but some have been re-estimated with after-the-fact estimates. The product backlog is similarly mixed: mostly the initial, before-the-fact estimates but some estimates that have been revised after-the-fact because of what was learned by developing previous user stories off the backlog.
Having both before-the-fact and after-the-fact estimates on your product backlog and list of finished work can cause a lot of confusion for the project. When all estimates are given in before-the-fact numbers we can reason about them and compare them. Suppose the team is estimating a new item and want to say its equivalent to 20 story points because it’s similar to another item that has been estimated at 20 story points. That logic makes sense if the original item has not been re-estimated. If the old item was given an estimate of 10 before the fact and re-estimated to 20 after the fact then it is harder to know if the new item should get a 10 or a 20. With the re-estimation having occurred we’re in the position of saying “Before I start this one I think it’s a 20 because the other one felt like a 20 after I did it.” That’s weaker than “Before I do either of these they seem the same size.”
So, does this mean you should never re-estimated? Absolutely not. There are times when you want to re-estimated. Generally re-estimating is useful when you completely blew it on the original estimate and can see that the mistake was a rare occurrence. (That is, if every estimate is systematically off by half I wouldn’t re-estimate.) Second, you should re-estimate when there has been a change in relative size. For example, the team has discovered that learning AJAX will be about half as hard as they thought. We’d want to fix that because the new knowledge tells us that our relative estimates are off-kilter for the AJAX-heavy stories.
Tags: agile estimation, story points
I have a doubt, the re-estimation can be done during the Sprint? or we have to wait until the next Sprint Planning?
Regards.
I would normally discuss the re-estimating at the sprint retrospective.
I’m glad you posted on this, we are on day two of an iteration and we just started a 1 point story (i.e. displaying a particular report). We now believe that it would be better to implement more of the underlying domain model to support the story, but this will significantly increase the amount of effort required to implement the story. Should we bump up the # of points for the story, or leave it as it is?
In general I do not recommend re-estimating. If you have one 1-point story you’re aware of that should be bigger, it is likely that you have others. If you re-estimate this one, you should re-estimate the others. But–do you know which they are? Very likely not unless you’ve discovered something about a specific class of backlog item. If you’ve learned, for example, that working with DB2 is harder than you thought when you did the initial estimates you may want to increase all DB2-intensive stories. Most of the time though it’s one story we botched and we don’t really know why. It’s better to leave those alone because there are others like it in the backlog but we can’t identify them.
This is a verry intressting topic.
Let’s say we have a Sprint with four stories. And let’s say we for some bad reason need to push Story 3 out of the Sprint, as we otherwise will fail the Sprint.
But work might have been started, maybe even be close to completion. Let’s say it was a 6 point:er that got close to completeion befor pushed out of the Sprint.
Should it not be re-estimated befor beeing assigned to the next Sprint?
I would drop that 6 pointer from this sprint. The team will be frustrated because they “don’t get any credit” for the work they did. Velocity is counted on an all-or-nothing basis so it will be artificially low in that sprint (let’s say by 5 points because they did 5/6 of the work but we’d never know that for sure). We leave the 6-point estimate alone and they do it in the next sprint. Velocity will be artificially *high* in that sprint. This is one of the reasons why I say that velocity will fluctuate from sprint to sprint and that we are interested more in long-term averages and trends in velocity than exactly what it was last sprint or iteration.
Also, if the team’s velocity was 100 in this in the first sprint (but was “really 105″), they won’t care about the 6 points. If their velocity is 4 instead of 9 they’ll care a lot. Doing it as I describe sets up a good incentive to keep relatively small backlog items *and* backlog items that are somewhat similar in size. Both of those are good things for improving the flow of work through the project.
Mikael , velocity by the very definition ( http://en.wikipedia.org/wiki/Velocity ) “is defined as the rate of change of position”. If story has not been competed, the Product Owner or customer’s position in relation to this story did not change. Therefore I would fully agree with Mike – a velocity component related to that non-completed story was indeed 0.
One of the reasons for it that that in agile methods and in Scrum in particular the customer’s benefits are “quantized” – customer can benefit only from the whole user story completed, e.g. he cannot and should not use something coded, but not really tested.
We run into this problem quite a bit, where we have several stories that are “nearly” finished in a sprint – often caused by the fact that the dev work on the stories took longer than expected, so several stories end up going to QA right at the end of the sprint.
I would agree that the points for those stories shouldn’t be added to the current sprint because they weren’t complete and therefore weren’t “delivered”. But, the question I have is about counting the story points for the continuation of the work on those stories in the next sprint.
If a story was taken off the backlog, added to a sprint, but was unable to be finished in the current sprint, then at the end of the sprint we add a new story to the backlog, but the size of this new story reflects only that work that remains. Our product owner pulls stories off the backlog for the next sprint based on priority and sizing of the stories in the backlog, and that list of stories may or may not include the new story (though normally it is included).
At this point, we are starting a new sprint and there isn’t a concept of a maintaining the “history” of work on stories from previous sprints. We simply look at how much work we have to do in the upcoming sprint and our sizing for those stories.
In this manner, the story points help us to understand how much we are able to “deliver” to our business users in each sprint. We don’t “carry-over” points between sprints if a story doesn’t get finished in a single sprint, because that doesn’t tell us how much we can get “done” in a single sprint.
It’s true that it means we don’t get all of the “credit’ for work that we do across multiple sprints, but it does help us to feel the pain of not completing work items within the sprint as estimated. Does this approach make sense, or does that seem wrong?
I think it’s important to remember why you’re doing estimates and not get too hung up on treating it as a game. The reason to do estimates is to allow you to (and the customer) to have a reasonable idea when the value is going to be delivered and to allow you, at the beginning of a sprint, to pull in an amount of work that you can reasonably commit to finishing during the sprint.
For deciding what stories to pull into a sprint, it makes sense to use whatever you know at the time you’re making that decision, so re-estimating at that point (as late as the sprint-planning meeting) makes sense. Re-estimating during the sprint makes no sense, because it’s not going to affect any decisions.
If you partially complete a story during a sprint, the right answer depends on whether you can split the story – if you can deliver potentially shippable value from the story, then it probably makes sense to divide the points between a narrowed story that you can treat as completed in the sprint and a new story that goes on the backlog for re-prioritization. If you can’t deliver any potentially shippable value from the story during the sprint, then I think you should push it back to the backlog, but re-estimate it based on the remaining work (so it makes sense when you’re estimating the next sprint). That reduces your velocity for the current sprint, but makes the next one correct.
Sandra–
I tend not to worry about velocity as a short-term predictor. It will bounce up and down. I get much more interested in velocity as a long-term predictor. I want to know our average so I know what we’ll finish in five more iterations. If I’m interested in average velocity it doesn’t matter if a team gets 5 points this iteration or next so the whole question of partial credit goes away.
By not carrying over any points you are understating your actual velocity. That will help your business learn not to overcommit but you won’t have a true feeling for your actual capacity over time.
Scott–
I agree with you that we should always use whatever information we have at hand when estimating or planning. When I do iteration (sprint) planning I don’t even consider the story point estimates. We grab one item from the product backlog, break it into tasks and hours, discuss whether we can commit and then repeat until full. The story points–as I use them–are for long-term planning. When we plan an iteration and take a lot more time to do so we do it by discussing tasks, hours and commitment.
I would like to get clarification on how story points should be used during Sprint planning. In the days before we began calculating the teams’ velocity, the team would commit to work until they feel they couldn’t bring in any more stories (based upon task estimates and team availablility). We found that the teams would over commit fairly significantly and weren’t able to achieve doneness for their commitments.
Over the course of the past few months, we jumped on the Velocity train and began computing Velocity for our teams. Over the past few sprints, we used the teams’ average velocity during Sprint planning and tried to not go beyond 10% of the average velocity unless it was a very sound circumstance. This sounded lovely in theory and were excited that this would provide the team with a more “realistic” commitment. This has worked well with some teams but has left others feeling frustrated. For example, if a story in previous sprint was worth 20 points and only lacked 5 hours to be completed, it skewed what the team felt they could commit to in the next sprint because we carried over the full 20 points. Also, this left teams pulling in unscheduled, tasked stories into the Sprint when needed but felt they couldn’t commit to doneness on them.
In summary, how should Velocity be used during Sprint planning?
Mike,
You say: “When I do iteration (sprint) planning I don’t even consider the story point estimates. We grab one item from the product backlog, break it into tasks and hours, discuss whether we can commit and then repeat until full.”
I have recently (the last sprint) instituted not looking at points during sprint planning for the purposes of making commitments but rather using a “commitment based estimation” for determining sprint capacity. That being said by-the-book Scrum calls for doing a sprint planning meeting with the Product Owner to ask questions about backlog items and commit to them followed by a Sprint Planning meeting for just the team where they decompose items into tasks. It seems to make more sense to commit after you’ve decomposed but how does that affect the Scrum lifecycle?
I’m curious why you aren’t using story points to do your sprint planning. I thought that the point of measuring story point velocity was partly to determine how much we can take on (or commit to) in a sprint. Do you only use story points for longer-term planning (e.g. release planning)?
Hi Sandra–
I’ve answered your question with an entirely new blog entry: http://blog.mountaingoatsoftware.com/?p=15
Hi Jimi–
I have to admit I’ve never understood how a Scrum team is able to make a very good commitment without having first broken the product backlog item into tasks so that they have a bit of insight into what’s involved. I’ve been doing commitment-driven sprint planning for years and believe it is the best way to do it.
I think the “by the book Scrum” that you refer to comes a bit from the time when teams were much more likely to do 4-week sprints than they are today, when they made commitments to more vague feature descriptions (“we’ll figure out a way to add an undo feature”) than to the more specific user stories and smaller product backlog items popular today, and when teams were more likely to user a “sprint goal” as a way of saying “Well we missed the exact product backlog item but we were successful because we achieved the essential part of the sprint goal.”
On teams or in an era when these approaches were more prevalent it may have made more sense to make commitments without knowing tasks. For my teams, though, I want the product owner present during sprint planning (or at least accessible for questions) and I want to do a rough task decomposition before I feel forced to commit.
Hi Misty–
In response to your question and Sandra’s I wrote a separate blog posting that I believe covers your question about the role of velocity in sprint planning. You can read that posting at http://blog.mountaingoatsoftware.com/?p=15
but essentially velocity plays no role in sprint planning other than perhaps as a sanity check after you’ve taken a commitment-driven approach to sprint planning.
Hi,
About Mike’s comment:
“Before I start this one I think it’s a 20 because the other one felt like a 20 after I did it.†That’s weaker than “Before I do either of these they seem the same size.â€
Having taking Mike’s training on Agile Estimating & Planning, I am confused. To me, the main motivation about re-estimation was to use posteriori knowledge, and reflect your estimations continuosly to your customer when to deliver value using min/average/max velocity.
There even an example section in Mike’s training on Agile Estimating & Planning. There, a priory knowledge estimation of UI stories were low, then one of them proved to be difficult with a higher size value. Then, all related UI stories’ sizes we increased at Sprint Retro.
Could you tell me if/where I misinterpreted this thread / the training ?
Hi Hakan–
My main advice on re-estimating is that as much as possible we want to be comparing apples to apples. If some of our estimates are created before we do the work (a priori) and some are created after we do the work (a posteriori) then we are mixing estimates of different nature. In one case we are saying “Before we start the work on this and after discussing it at a high level we think it’s an 8.” In the other case for a different product backlog item we are saying “After having done this and seeing how hard it really was we now think it’s an 8.” Those eights are not the same.
Yes, I think it’s OK to re-estimate. Practically, though, it happens pretty infrequently with the teams I’ve worked with over extended periods. They’ll re-estimate a story here or there every few sprints but re-estimating that little doesn’t significantly help or hurt the project. Any time we finish a story and realize we really blew it on the estimate, we should re-estimate that item. But we should also wonder: Are there other items in the product backlog that are similarly incorrect? There almost certainly are. So ask yourself what was unique about the re-estimated story that made you re-estimate and what other stories seemed similar before you worked on it and might therefore be estimated with the same error.
Hi,
I’ve been doing almost the same Mike said he does.
I do Product Backlog estimates in story points, I than break the stories selected for the sprint backlog into smaller tasks. And after that, team members estimates, in hours, the length of each task.
But, during the sprint, i. e. everyday, the team members re-estimates the number of hours they think they will need to get the task they are working on done. And that reflects on a updated Burndown chart. It enhances the sprint visibility as it makes visible the estimates deviation and enable early decisions.
So, Mike, as you said your teams estimates the tasks on hours as well, do you re-estimates the remaining hours to complete the task too? If you don’t, do this deviation not affect the accuracy of your Burndown and can let to misinterpretation of it?
Thanks,
Pedro Reys
Hi Pedro–
Yes, every day each member is expected to update the estimates of any sprint backlog tasks for which they now know more. Generally, of course, estimates should go down but that won’t always happen.
I don’t see why you think this might lead to an accuracy problem with the sprint burndown chart. The sprint burndown chart should always show the effort remaining. That will happen if the team members update the estimates (in hours) as they learn more about what is left on a task.
If there any anomalies in the shape of that burndown chart, it may not be pretty but it will be informative. A wildly gyrating sprint burndown may indicate that the team doesn’t really understand the work, for example. Perhaps this anomalies in the burndown are what you mean by deviations. Seeing this is a good thing because it provides information to the team and their ScrumMaster.
–Mike
Hi Mike,
I totally agree with you. And I’ve been doing exactly the same way as you describe and it is working pretty well.
“I don’t see why you think this might lead to an accuracy problem with the sprint burndown chart.”
Actually, I dont think it at all. Please, forget that ugly last sentence.
Thanks Again.
Pedro
Hi Mike,
I’m the product owner of a company in Spain (Also I have the scrum master certificate
)
First thing you need to know is that in our company, in general we decided no to change story points of the stories after they have been estimated during the sprint planning. We believe the important thing is the average and the option of doing it to all PBI we believe would be a waste.
But in the last sprint we had a situation that has been resolved in a way which I don’t think it was right.
Two days after sprint planning the scrum master comes to me and says that one of the stories (upgrading a third party components suite to the next version), required to build a new skin for our product (because this third company made an exceptional change in their components). So far, in the past, this story for updating the suite always has been a two or three storypoints (last 6 times we have done it more or less in that), but in this ocassion, really it has been more than 13 !
When situation is detected, Scrum master says that since this is a lot of work , if we continue with this story some other stories will have to be taken out from that sprint (finally about 12 story points were taken out). Original This story , at the beginning of the sprint, it was estimated in 3 storypoints.
The question:
-Given that this has been excepcional comparing to the stories in the past for updating this component, and
-Given that we had to take out stories from the sprint.
Don’t you think that this story should have been re-estimated to change the story points to what really was needed it or at least, change it to the number of storypoints that were taken out from the sprint ¿??
From my opinion, I clearly think that since the team and scrum master decided not to do it, velocity has been extremely altered.
Thanks and best regards !
Hi Carlos–
The biggest problem I read here is the ScrumMaster telling the team they must add a user story to the sprint. The ScrumMaster should be protecting the team from mid-sprint changes like this, not causing them. If you are the product owner for that team, you should have been the one interested in this new functionality, not the ScrumMaster.
The ScrumMaster should also not have been the one to demand that other stories be removed from the sprint. That should be up to the team.
And, yes I would have put a number larger than 2 or 3 on the story in this case. But I wouldn’t have considered it a re-estimate. What I’m reading above is a new change request that at the surface is similar to 6-7 past changes that were 2-3 points each. But there are other factors that make this one harder so I would estimate it higher since it will take longer. I’d view that as the first estimate given to this functionality not a re-estimate.
Hello again,
Thanks for your answer.
Just one thing that has been missunderstood in the explanation I made (probably my english it is not good enough).
The scrum master didn’t tell the team that a new story should be added. The thing is that in order to do the story (updating the third party controls), because a technical change in the new version of said suite of controls, after a few hours of starting do work with the story, the team found it was absolutly necessary and a must to build a new skin for our product.
Because completing that story was a lot of work and therefore other stories were need to be taken out from the sprint, that’s because scrummaster toldme that if I wanted them to continue with that story, others should be taken out from the sprint (but was in name of the team
)
The fact was at the sprint review, the team did not agree to change the story points for that story.
Thanks,
Carlos.