Comments on: Should Companies Measure Productivity in Story Points / Ideal Days?

By: Mike Cohn

Mike Cohn — Thu, 11 Mar 2010 20:54:02 +0000

Nate–Can you provide a page reference to “Agile Estimating and Planning”?

By: Nate

Nate — Thu, 11 Mar 2010 20:46:30 +0000

I have a question on one of your earlier comments. You state that it’s appropriate for a story to be sized (in part) based on current knowledge, i.e. establishing a direct link between competency with a technology and the size of a story.

In “Agile Estimating & Planning”, you appear to state the opposite when making the case against ideal days, going so far as to specifically use an example of a developer who isn’t experienced with a technology up front to prove the point. Perhaps I’m misreading one or the other, but these appear to contradict each other. Can you provide a bit more clarification?

By: Mike Cohn

Mike Cohn — Wed, 06 Jan 2010 02:26:19 +0000

Hi Lanooba-- I have a chapter on metrics in my Succeeding with Agile book. There is also a good paper on Agile EVM by Tamara Sulaiman on InfoQ.

By: Lanooba

Lanooba — Wed, 06 Jan 2010 02:21:49 +0000

I’m joining the discussion a little late; and I hope it’s still open. I’m being directed to provide PMI-style metrics for agile projects (i.e. earned business value, CPI, etc). While I’ve read the PMBOK and am familiar with the concept, I have difficulty applying it to agile.

Do you have any thoughts/recommendations?
Are these types of metrics even valuable?

By: Mike Cohn

Mike Cohn — Thu, 17 Dec 2009 02:27:19 +0000

Hi Greg–
Great example. We might call it gaming the system but we might also call it getting the team to behave in a better way. Most simple metrics like this work very well for a short period of time (as in your case) and then we move on to other metrics.

By: Greg Reynolds

Greg Reynolds — Thu, 17 Dec 2009 01:08:36 +0000

I think we measure what it makes sense to measure in terms of what will deliver business value, for as long as it delivers value. One team I worked with was having difficulty with over-committing. At the end of each sprint we had an unacceptably high number of either incomplete or partially complete stories. What I ended up doing was establishign realistic, sustanable goal with the team, then adding Plan vs. Complete as a percentage on a scoreboard. We monitored it for a number of months until the team brought their plan vs. complete into an acceptable range.

Did they ‘game” the system? In a manner of speaking. They began committing to a bit less work each iteration until they reached a level where they felt it could sustainably be delivered. After a couple of months at this level we no longer needed to track this metric on our scoreboard, as the team had realized better planning and commitment practices, which was the goal in the first place.

By: Mike Cohn

Mike Cohn — Tue, 22 Sep 2009 02:01:26 +0000

Hi Z:
Work should always be estimated relative to the current state of the world–your current knowledge, the current state of the system, etc. Think about the opposite situation: What if the application had become fragile and hard to maintain. Would you want to estimate it in points with the assumption that the application is as it was? (Nice, clean and maintainable 4 years ago?)

Also: Could we even do this? I think it’s hard enough to estimate how long it would take right now to do something. But to estimate relative size based on old assumptions would be even harder.

By: Z

Wed, 09 Sep 2009 17:16:21 +0000

Hi Mike,
Following on the above thread, would it be acceptable for story estimates to decrease because of good design/architecture decisions the team made on older stories? The stories would be easier to complete and the design/architecture is already laid out. Yet, if you do decrease the points, then you’ve done away with relative sizing and you are now completing more stories at a lower velocity. For me, I would like to keep the points the same and simply increase the velocity.

By: Mike Cohn

Mike Cohn — Sun, 15 Mar 2009 23:46:31 +0000

Hi John–
Yes, an item estimated a year from now after a team jells or gets good with a technology would have a lower estimate. Picture a team that has learned Ruby over the past year. They might say, “This is only 2 story points because we’re good with Ruby now.” A year earlier they might have called that 10 because of the fumbling around they would have done while unfamiliar with a new technology.

To ask a team to estimate otherwise is impossibly difficult: you would be asking the team to abstract away the current state of the world and to estimate back to whatever level of knowledge/experience/etc they had don the first day of the project. This is impossibly difficult for a person who has been on the team the entire time. It’s flat-out impossible for someone who joined the team mid-project to estimate back to what things were like before he or she joined the team.

By: John Esser

John Esser — Sun, 15 Mar 2009 07:09:17 +0000

Mike-
I didn’t clarify this in my post, but we estimate relatively as well. One of factors we consider is “complexity,” but several others as well including “does the code need to be refactored?,” “how many modules we will code?,” “how much will need to be done to test it?”, etc.

After reading our posts again, it seems that maybe we’re really doing the same thing! It’s just that we have different ways of looking at the essence of story point. In either case we both want a “size” value and we’re both trying to get each of our teams to NOT talk about time during the estimation activity! (It’s so easy to do…sigh)

My only remaining question is this: if the team’s perception of estimation is rooted in “relative duration” and that is factored into their estimate, then as the team gains proficiency and domain knowledge, or just “gels,” wouldn’t stories tend to be estimated smaller over time?

This would seem to create a system that has a relatively constant velocity, but having more items getting done per sprint. Whereas in mine, story items are consistently sized, but the velocity increases as the team becomes more proficient.