As a measure of the amount of work completed in an iteration, velocity works extremely well when teams are relatively stable. If the same people stay on a team, it is reasonable to assume that the amount of work they complete will be relatively constant from iteration to iteration. This allows us to plan using inferences such as “This team has an average velocity of 25 points per iteration over the last year and they have time for 8 iterations in this new project; therefore they will complete around 200 points in those 8 iterations.”
But what do we do when we are managing an agile project where team membership or size changes frequently?
To answer this question most effectively, you should collect data on how teams of different sizes have performed over time in your organization. When I was a VP of Development at a couple of agile organizations, I used to collect data on velocity and team size changes in a simple spreadsheet similar to this:
| Initial Team Size | New Team Size | Median of Last 5 | Iteration +1 | Iteration +2 | Iteration +3 |
|---|---|---|---|---|---|
| 6 | 7 | 25 | 20 | 24 | 28 |
| 6 | 7 | 16 | 16 | 15 | 19 |
| 8 | 6 | 50 | 40 | 40 | 42 |
| 7 | 8 | 12 | 10 |
The first column represented the size of the team before a change occurred. The next column represented the new size of the team (up or down). The third column was what I considered to be a reasonable “long term average” velocity for the team at its initial size. Because team’s could change frequently (by a person or two) I settled on using the median value of the team’s last five iterations. The tradeoff in using a longer measure (median of 15 iterations perhaps) is that you’d have fewer observations. The next columns represent the actual velocities of teams over the next three iterations. Notice that for the last team values are not shown for the last two iterations. This is usually because the team size changed again. If you have a significant number of teams, the rows in this type of spreadsheet will accumulate quite quickly.
In some of the organizations where I used this approach we did not have a standardized definition of story point (or we’re using ideal days, which were not as normalized as you might think). So all analysis was done on a percentage basis. What I wanted to know was “What is the average impact of adding a person to a seven-person team?” I would have loved the answer to be something like “Velocity goes up 15%.” Unfortunately, it wasn’t that straightforward because velocity often dipped for a couple of iterations before going up. By tracking data I found that usually by the third iteration a team had settled in on a new velocity, which is why my spreadsheet above only tracks through Iteration+3. By all means, track more and see what you find. (But keep in mind that the data will get sparse as team sizes will change again.)
Another tab in my spreadsheet, expressed all the data in percentage terms. The first two rows
| Initial Team Size | New Team Size | Iteration +1 | Iteration +2 | Iteration +3 |
|---|---|---|---|---|
| 6 | 7 | -20% | -4% | +12% |
| 6 | 7 | 0% | -6% | +15% |
(Example: Iteration +1 in the first row is -20% based on the team dropping its velocity from 25 to 20.)
I then simply averaged these percentages for each team size change to get results like:
| Initial Team Size | New Team Size | % Change in Iteration +1 | % Change in Iteration +2 | % Change in Iteration +3 |
|---|---|---|---|---|
| 6 | 7 | -10% | -5% | +15% |
This allowed me to answer all sorts of questions, including:
- What will this team’s velocity be if we add two people?
- How soon could we get this project if we added a person to each team?
- If I want all those projects done by the end of the year, how many people would we need to add?
- What would be the impact of not approving the new employees in the budget?
- What would be the impact of a 15% layoff?
There are of course many flaws with this approach. Adding Susan to the project is very different than “adding an unknown person” to the project. Still, if I have the data on averages across the board in our organization I can make assumptions about specifically adding Susan (if I want, there can be many more risks in doing this). Notice that the approach does not attempt to take into consideration who it was that was added or even what skillset the person had. You could collect such data if you wanted. As anal about collecting all sorts of data like this as I am when I have access to it, I knew though that collecting that type of data would have made this just hard enough that I wouldn’t have done it regularly.
I did collect a few other bits of data that I left out of the initial table (so as to have more horizontal room for the data of real interest). For example, I collected data such as iteration length and the name of the team’s ScrumMaster (the latter was in case I had questions a few weeks later).
The approach described here was just simple enough that I could get empirical evidence of the impact of team size changes. This was invaluable when discussing headcount changes with product owners and the CEO.
Tags: agile release planning, metrics, velocity
Hi Abby–
That’s a good idea. I haven’t published the data I have on percentage productivity changes because I only have it from two companies and I wouldn’t want anyone to take it safe to use industry-wide. Also, the two companies were ones where I was the VP of Development so I set things up pretty similarly, meaning I would expect the data to be consistent across those companies. I don’t know if the percentage changes I saw there would apply in companies with more or less corporate overhead, etc.
An industry-wide effort to collect this type of data would be wonderful. I haven’t seen anything along those lines, though. But, if anyone reading this starts collecting this type of data, please send me a copy and I’ll maintain a cross-company set. Hopefully we can then share the results with everyone with solid information on relative performance as team sizes change.
Thanks for the idea.
Oh, this is awesome. Thank you for this and the last post – both of which addressed questions I was having about shared story points.
It’s so great that you keep metrics, this is something we need so badly in our industry so that we can be smarter about things like the impact of adding a person to a team (I mean, really, there shouldn’t be any excuse for our not knowing that productivity initially dips, but we’re all hopeless optimists I suppose and not heeding Brook’s message!). Wouldn’t it be wonderful if there was some sort of centralized location that people could go enter metrics like these from their own projects and then some industry averages could be viewed? Have you ever seen anything like that?
I like this empirical approach and it proves that adding a person initially impacts velocity negatively (contrary to what people might expect).
The examples given were for 1 person. I’m curious: Do you have data for adding 2 or more people? I’m wondering if this is scalable or if adding more people has more of a destabilizing effect on the team.
Hi Rene–
Yes, I kept data on every team change we had. There were lots of 1-2 person (at a time) changes and occasionally a team would change by 3 people (up or down). If a team changed by more than that I didn’t track the change because if you change 4 people, you’ve pretty much created a whole new team (even if it was 9 people to start with).
Please note that the data above is not meant to represent real results you should use directly. I advise tracking this data in your company for awhile and building up your own database of velocity changes and team size changes. You’ll quickly build up enough data that you can start using it in the ways described.
Yes I have been collating data in a very similar way to help with the sprint planning process and more importantly to measure if the development process improvements we are making is having a positive impact on the team velocity.
I have found the best use of the story point averages are within the Sprint Planning meeting where they are used as a reality check to avoid over zealous teams signing up to too much work. The difference I have seen in taking this approach rather than just to base the next sprint on the previous sprint, is that teams are completing their work within the sprint period more and more.
How would you predict sprint velocity when the amount of vacation, holiday, and sick time fluctuates from sprint to sprint AND resources are added? For instance, one sprint may span a holiday and include 5 days of vacation time while the very next sprint has everyone in attendance plus a new resource in the sprint. I understand this is where an average velocity calculation can help. But a 5-sprint average will take 10 weeks to acquire (we have 2-week sprints) which is too long to wait in an agile environment. Suggestions?
Hi John–
I normally make the assumption that sick time will be constant from sprint to sprint. For vacations, if I’m looking ahead 3-6 months I will normally just assume that I don’t know when vacation will occur but that it is already accounted for in how I use a range of velocities to make predictions.
For holidays, if it’s a simple one-day thing I usually don’t worry about. If it’s something like Christmas or the US Thanksgiving holiday that are multiple days and seem to have a bigger effect because people are less focused, I do try to account for them. It’s October 25 as I’m writing this. If I were putting together a 3-month plan right now I would probably assume a slight drop in velocity around late November and late December/January 1 to account for this.
I’d either wing it and guess at how much to drop velocity by OR (even better) I’d look at what happened last year. I constantly tell companies to capture this type of data. Wouldn’t it be nice right now to know very simply that velocity dropped by an average of 25% for the sprints of comparable length that included Christmas last year? I wouldn’t make adjustments for team size or how many days of planned vacation were in those sprints, etc. I’d just want to know what the average drop was in percentage terms by sprint length.
If I were predicting velocity for a team that has never worked together and had only 1-2 sprints of history I would express my estimate of velocity as a range. I would use whatever historical data I have (from the 1-2 sprints) and would look at standard deviations of velocity from all other teams and apply that plus a good dose of my own common sense and intuition. But, even better, I would try not to be in a position where someone needs an absolutely perfect date from me for a team that has no prior history. I’d be willing to come up with an answer but I’d want them to understand all the potential problems with it.
This article probably assumes that team members are same and some addition and deletion of members is happening. If we keep the team size same but change the team members then we can see a very different velocity. How do you track team member changes for planning purposes? Practically, it is very common to see team members getting changed.
I have normally not kept the team size constant but changed members (that is, swap member A for member B). So in my experience there is usually a team size change as well. In developing this approach years ago I was after a general way to make predictions about changes in staff size, not changes based on specific individuals. I would suggest that your knowledge of the individuals will probably be the best way to guess their impact on velocity (“Hmm, better programmer but horrible personality, maybe a slightly negative impact overall.”). I’ve never tried to quantify that.
Hi Mike.
About team velocity, it measures how much of the backlog (story points) the team can consume (produce working software) in a iteration, right ? I have a question about how to compare velocities across different teams in different projects. Since story points are relative, velocity is as well. Ex: Team A in project I can consume 10 points per iteration and team B can consume 10 points in project II. Looking at the velocity it seems that both are equally productive, but in a further investigation you realize that team A used a very small story as base ( attributed 2 story point to it), and team B used a larger one. Considering this scenario, how can I compare velocities among projects? Even more, what kind of metrics I can collect and compare across projects?
Thanks a lot,
Diogo
Diogo–
Please look at these blog postings that address your question:
Establishing a Common Baseline for Story Points
and
Is It a Good Idea to Establish a Common Baseline for Story Points?
[...] this blog post Mike Cohn estimates that adding a seventh developer to a six man team will reduce the velocity by [...]
Hi Mike,
First of, I gotta say, I am a huge fan, I love your book and now I am loving your blogs.
I am currently facing a scrum issue at work, in regards to what is the best way to handle team members going to another project, but tracking the velocity as accurately as possible.
Here’s the scenario:
We have Team A, B, & C (scrum teams), their primary purpose is to work on new features; we also have team x (waterfall team) that is working on the final regression of the software product before it goes to production. However, team x needs more resources to finish by the deadline as well as need the knowledge and expertise of some of the folks on Team A, B & C. Team A, B,& C will lend team X a few folks, however, this now changes the resource count and velocity.
(Here’s the dilemna)
Some of the PMs feel that Teams A, B & C should still count the work that the loaned out members do for team X on their back log, so this way the resource count and velocity can be tracked and they feel that it would be accurate.
So the stories for Team A’s sprint backlog look like this for example:
Tasks Mon Tues Wed Thurs Fri
Code interface (team A task) 4
Test interface (team A task) 8
Add error logging (team A task) 4
Test languages (team X task) 4
Stabilize the Eval Environment (team x task) 4
Write the xyz class (team A task) 8
I on the other hand proposed your method on how to track velocity with a varying amount of team members from sprint to sprint, but I can’t get them to see that the impediments on team x are much different than the ones that team A experiences and can toy with their velocity numbers and make them inaccurate. Also, team X doesn’t track velocity, so management feels that we should track team velocity whenever we can, even if it means adding another project’s story to their backlog, just so the number of resources can stay the same.
The feedback that I am getting from managers is that there is a lot of overhead and no benefit to combining the 2 projects into 1 backlog because the loaned resource (from Team A for example) still has to:
• provide status to their team as well as Team X
• still go to team A’s planning meetings
• estimate stories for team A as well as provide estimates on the stories that they will do for team
• Enter their time into Team A’s scrum tool for the hours that they did worked on the tasks for team X
The way we use to do it, was treat the loaned resource as “time away from their project” so they can go work on the other project 100%. This would also allow us to use your method to track velocity for a varying team.
So I was hoping to get more feedback from you as to why 2 projects should not be combined into 1 backlog. Amazing there isn’t much information about this on the internet, I have tried to searching and I believe you are the only one who has at least offered up a solution to track varying resources. Feel free to let me know when there is a good time to track stories from 2 different projects in backlog as well, because I can’t personally think of any reason.
I apologize for the length of this, but would appreciate any advice that you have to offer.
Thanks,
Krystal
Hi Krystal–
Whoa, way too much there for me to perhaps comment on fully but I’ll try.
We call it a “product backlog” because there should be one per product. It’s not a project backlog or a team backlog. To have team backlogs would create prioritization problems (team A’s backlog could be entirely lower priority than team B’s but we wouldn’t know it if they were separate). I advise a single product backlog with multiple views into it (essentially MVC for backlogs). Some tools support such an approach.
The real answer for your situation though is that when someone on Team A works for Team X (the waterfall team doing testing) that work should not count in Team A’s velocity. Team A gets credit presumably for the points on the stories they “finish”. But those items aren’t really finished until they pass test (by Team x). So giving Team A credit for the story in sprint 3 when they “finish it” and then some more credit in sprint 4 when it’s tested is wrong. It overstates velocity.
It’s easy to see why: Suppose Team A does really sloppy work but claims to be done. They get say 5 points. Next sprint team x finds a ton of bugs and team A fixes them, claiming 3 more points of credit. Contrast that with Team A doing an awesome job and having no bugs found by Team x. In that case Team A would take no points in that second sprint. The sloppier Team A is, the more velocity points they get. Bad idea!