Let’s consider that perspective for a moment. How many significant things in your personal life would do without at least some planning first? I doubt you would plan a wedding, a relocation to another city, a holiday trip, or any such event without first engaging in some amount of planning.

Suppose you are considering a first-ever trip to Italy. You would plan that–which cities? how long in each city? what’s your budget? and so on would be some of the questions you would consider. Now suppose you are planning your one hundredth return visit to the city in which you grew up. You will even plan this trip–even if the extent of that planning is to decide you don’t need to plan at all.

Planning is the act of thinking about the future. Sometimes that future holds risk and uncertainty. In those cases we plan more than when the future is highly predictable as a hundredth visit to your childhood home town would be. When a future activity is highly predictable, planning may consist of nothing more than a few milliseconds of rejecting the need to plan further.

Of course on a software project it is rarely this simple.

What about estimating? Do we really need to estimate? Yes, because estimating is a pre-requisite to planning. You cannot plan without estimates in mind. Those estimates may be very informal and very implicit. As I write this I am on a flight to California. Before boarding the plane I got cash from an ATM. I estimated my upcoming need for cash to do that. $200 should do it. That estimate took less than a second and I was perhaps not even conscious of it, but it was made.

When a product owner says, “I’d prefer to add this feature rather than that feature,” the product owner is acting with at least some implicit estimate (perhaps guess) of how long each will take. When a programmer chooses late in the day to fix a bug rather than start a new user story before going home, that programmer has made an implicit estimate that fixing the bug fits better with the hours left in the day.

Teams that say, “We won’t estimate. We’ll just make every user story the same size,” are estimating. They are estimating that this user story is the same effort as all other user stories. I’d even argue that it’s harder to make each story the same size than it is to use a small range of effort sizes on various stories.

These estimates must be made. Yes, they can be subconscious but they are made. Those who blog and post to newsgroups saying “estimating is waste, don’t do it” are ignoring these types of estimates.

But are these casual, perhaps subconscious, estimates OK? Wouldn’t teams be better with formal estimates?

Perhaps but not in all cases. A team should estimate and plan only to the extent that further investment in estimating and planning will lead to different actions. If you will do the same thing even if you estimate or plan more, stop. But if further planning is likely to lead to better decisions (more confidence in a delivery date, better prioritization of functionality, or so on), then estimate and plan further.

As an example, we recently added quite a bit of functionality to this website to support our new eLearning course on Agile Estimating and Planning. I did not ask the programmer who did that work to give me more than cursory estimates. I’m still enough of a programmer to have an idea how long the new features would take. I’ve worked with him long enough to know how fast he is. Asking him for detailed estimates would not have changed anything about that project.

So estimating and planning are necessary. They can (and should be) lightweight. You should stop when further planning is not likely to lead to improved decisions worth the extra effort.

The entire course is now available on the Mountain Goat Software site. Check out the free previews. The course can be purchased for streaming or for streaming + download. We also offer group discounts and an innovative, easy way for someone such as a manager, ScrumMaster or similar to buy and distribute licenses to team members.

The 44 videos are interspersed with 9 interactive quizzes. If you miss a quiz question, you get immediate feedback telling you what the right answer is. This approach may not hold up to some certification organization’s rigor, but it’s sure a helpful way to make sure you’ve learned the topic. You can see an example below.

The course qualifies for 4 PDUs from the Project Management Institute and is perfect for PMPs or aspiring PMI-ACP candidates. At the end of the course, you earn a Certificate of Completion as proof of your participation.

Overall this course has been in development for 16 months and we’re hopeful you’ll be able to see the attention to detail we put into it.

I hope you find this news as exciting as I do. Please help me out by spreading the word to your friends, coworkers, grandmothers, and anyone else who might be interested. Thank you.

Resampling is based on the idea that things we’ll observe in the future will be similar to the things we’ve observed in the past. In the examples below we’re saying the velocities a project team will see in the future will be similar to ones that occurred in the past. Resampling works by imagining we’ve put all old sample velocities into a bag. If we have past velocities of 18, 17, 18, 19, 22, and 20 imagine each of those written on separate slips of paper and dropped into a bag. Note we’ll have two slips of paper with “18? because our Agile or Scrum team here had that velocity twice.

To predict future velocity we reach into the bag and pull out one piece of paper. What’s written on it is our prediction of velocity in the first sprint. To predict the team’s velocity in the second sprint, we reach into the bag and pull out another slip of paper. But, before we do that, it’s important we put the first slip of paper back into the bag. This is called “resampling with replacement.” We want to replace because for any given sprint the team is equally likely to get any of their past velocities.

Suppose we’re trying to predict how much work a given team can complete in a coming ten-sprint period. We would resample (with replacement) ten times. Each time we’d note the number we pulled from the bag. After we pull the tenth slip, we would sum the ten values we pulled and that is one possible result for this team over the coming ten-sprint period.

It’s quite possible we could get lucky and pull the highest-valued slip (22) each of ten times. Or we could pull the worst-case slip, 17, each of the ten times. But these are unlikely. If we could in some way in the real world run the project hundreds or thousands of times, we would occasionally see the team repeat their highest (or lowest) velocity every sprint but it wouldn’t happen very often.

Since we can’t run the project hundreds or thousands of times in the real world, we want to simulate doing so on a computer. We can then learn a great deal from the results. For example, suppose we are ready to start a project that will have ten sprints. It would be helpful to know things like:

• what is the average amount of work completed over those ten sprints?
• what percentage of the time does the team finish more than say 200 points of work?

It’s actually quite straightforward to answer these questions using simulation and resampling. Let’s see how it’s done. You can follow along with this velocity resampling spreadsheet. In the figure below (from that spreadsheet), cells B3 through B28 show historical velocity.

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Because our hypothetical project here involves ten sprints, cells D3 through E12 show the sprint numbers (1-10) and a resampled velocity for each.

Selecting a resampled velocity is simply a matter of randomly selecting any velocity from the B3:B28 range (our 26 historical velocities). This is done with the formula:
=SMALL($B$3:$B$28,INT(COUNT($B$3:$B$28)*RAND())+1)
which generates a random number between 1 and 26 and then uses the SMALL method to select that item from the list of values. (Note: SMALL selects the nth smallest item from the list of values; we could have used LARGE instead. The goal is just to randomly select a value from the list of velocity in B3:B28.)

Because we’re using the RAND() function in E3:E12, any time you change any cell in the spreadsheet, the values in E3:E12 change. This is a desirable side effect of using RAND().

E3:E12 simulates one run of ten sprints. We’d like, though, to simulate 100, 200 or even 1000 runs of the project (each with ten sprints in it). To do this is slightly tricky because we’re going to use something a lot of people aren’t familiar with in Excel: a Data Table. In our spreadsheet, the Data Table is in G3 through H202, a portion of which is shown below.

The G column shows a sprint number, the H column shows a sum of ten velocities, representing one ten-sprint project in our case. In the example, in the figure at left you can see that the first simulation of the project yielded a total of 230 points done in the ten sprints of the project. In the next row (cell H4, but labeled ’2′ in column G of the spreadsheet), the team got a much higher simulated velocity, 264. In the spreadsheet, I repeated this 200 times but you can do more or less as you prefer.

For brief instructions on how to create a data table, see the end of this post. For more complete instructions, see the Excel documentation.

Armed with 200 simulations of the ten sprints of the project (or ideally even more), we can now answer the question we started with, which is, How much can this team finish in ten sprints? Cells E17 and E18 of the spreadsheet show the average total work finished from the 200 simulations and the standard deviation around that work.

In this case the resampled average is 240 points (in ten sprints) with a standard deviation of 12. This means our single best guess (50/50) of how much the team can complete is 240 points. Knowing that 95% of the time the value will be within two standard deviations we know that there is a 95% chance of finishing between 240 +/- (2*12), which is 216 to 264 points.

If my boss wants a guarantee, I might say, “We can pretty much guarantee 216.” Technically I know the math doesn’t support the guarantee. There’s about a 2.5% chance we fall below that. However, humans are involved and just about any good team I’ve been on would be happy to kick in some extra effort sometime over the ten sprints to finish the 216 we committed to rather finishing with 210 if that 2.5% chance does occur.

Another interesting issue we can address with this type of simulation is the boss who says, “I need you to get 250 points done in the next 10 sprints.” You can see how likely this is to occur by scanning down the resampled values (the H column) and seeing how often the value there equals or exceeds the value the boss, client or customer wants. The spreadsheet is set to do this automatically as shown in this figure:

Type in the total number of points desired in L20, the spreadsheet will see how many times that number or higher occurred in the simulations (L21) and then report it as a percentage (L22). In this example, if the boss wants 250 points in 10 sprints, we can reply that while the team will try to achieve that, historical data shows that there is only a 20% chance of that occurring.

Hopefully you’ve found these examples of working with resampling to simulate projects helpful. There are many more things that can be done using this technique. I’ll provide additional examples in future posts.

You can download the velocity resampling spreadsheet used in this examples.

Note on Creating a Table Table

To create a Data Table, in cell H2 (the cell above where you want to put the simulation results), enter the formula you want to calculate as a result of each simulation. Because we’re interested in the sum of their ten sprints worth of velocity, I’ve entered:
=SUM($E$3:$E$12)

Next, if you’re interested fill the sprint cells (the G column) with numbers from 1 to 200. Then highlight cells G2:through 202 (assuming you want to do 200 simulations as I’ve done here). Notice you’re highlighting starting one row above our simulations through the last row of simulations. Now create the data table. In my version of Excel (Mac 2011), I do this by selecting Data | What If | Data Tables. You’ll get a little dialog box asking for the row or column input cell. Move the cursor to Column Input cell and select G2. Close the dialog and you’ll see the 200 rows fill with simulation results.

Each column of data and pair of bars shows data from stories of the size given in the Points row. The first set of data is for one-point stories, the second set of data is for two-point stories and so on. Looking at the one-point data, we see that the median number of hours to complete a one-point story (at this company) was 21 hours. The shortest (From) took 6 hours and the longest (To) took 36 hours. The shortest and longest are shown in the green and blue bars above the table. The median is graphed as the red line.

Let’s look at the two-point stories. We see there a median effort of 52 hours and a range from 31 to 73. If we assume the 1-point stories were perfectly estimated, we would expect 2 point stories to have a median of 42 instead of the 52 we see here. Or perhaps the 2-point stories were done perfectly and the 52 is right. In that case, the median for one-point stories should have been 26. Most likely, neither is perfect and the “perfect” estimates are somewhere between. Either way, the team has drifted a little bit because the median of the 2-pointers does not equal twice the median of the 1-pointers.

But let’s look at the three-point stories. Three-point stories should presumably be triple the median of the 1-point stories. The median should be 63 and we see that it is 64. Wow. Very close.

The five-point stories should presumably have a median of 5×21=105 and they come in at 100. The eight-point stories should presumably have a a median of 8×21=166 but they only come in at 111. But, hold on, we have to make a bit of an adjustment in this thinking for the 5- and 8-point stories. If you use these numbers the way I recommend, you know to think of them as buckets. An 8 is a bucket that holds all stories from six through eight points in size. That is, if you have a story you think is a six, you can’t fit it into the five-point bucket so it goes into the eight-point bucket.

This means that our five-point stories are really fours and fives. If we have the same number of each, the average size of a story in the five-point bucket is really 4.5 points. Multiplying 4.5 points x 21 hours (length of the 1-point story) give 94.5, off by about 5% from our measured median of 101 hours. Not so bad.

Since the 8-point bucket holds stories of size 6, 7, and 8, we can assume the average story given an 8 is really worth 7 points. And 7 points times 21 hours = 147 hours. The 8-point stories should have a median of 147. They don’t, they have a median observed value of 111. So this team has definitely drifted by the time they reach their 8-point stories.

(Let me add a sidenote here that rather than simply assuming the 1-point stories were perfect and using their 21 for all this mulitplication, a better approach would be to do linear regression analysis, which can be done with Excel. You can then look at the r-squared value to see how well the values fit. But, I didn’t want to go through all that math here and with data like this, we can, I believe, see that things work out pretty well up to 8.)

I encourage you to think about collecting data like this at your company. You need to be careful though. Because you’ll be collecting actual effort expended on each user story, it’s possible that team members feel more than the normal amount of pressure to finish within any estimates they give. They may then respond by padding their estimates. This defeats the whole purpose. So, show a graph like the one above to the team and be clear that having this data can help them. For example, the team above could learn that they put 8s on stories that should perhaps have been 5s. (Looking at the data, they could also learn that they estimated correctly but that they really had more sixes in the eight bucket.)

Most teams who do this will find that they are good through about 8, as in this example. With some awareness of data like this and some additional practice and calibration, just about any team can get good across a 1-13 range. Beyond that is tough and numbers above 13 (and possibly starting with 13) should be used with caution or only for answering rough, long-term questions like, “Is this project a couple of months or are we talking about a year to do it?”

If you collect data like this for your teams, I would appreciate it if you would share it with me. You can just email it to mike@mountaingoatsoftware.com rather than posting it here. I’ve been working on some analysis of data like this and the more teams I have, the better.

My first bit of advice is always to try to put off answering until you’re able to get even one sprint of historical data. But that’s not always possible. When it’s not my general recommendation is to conduct one or more commitment-driven sprint planning meetings and use those to forecast likely velocity.

However, some people are still more comfortable thinking in hours and rather than forecast velocity, they want to forecast the number of hours a product backlog will take. From there they will usually estimate the duration of the project (something that could be done more directly with a velocity estimate). But, since it’s a common question, I’d like to address it.

Here’s essentially the question I was asked. I’ve paraphrased, simplified and left out extraneous information:

We have no historical velocity data. We have a product backlog of 300 user stories. Each user story has been estimated, most in the range of 3-13 points. Would the following approach be reasonable:

1. Grab a random sample of 40 stories.
2. Break each of those stories into tasks and estimate the tasks.
3. From the task estimates come up with an average number of hours per story.

For example, suppose the randomly selected 40 user stories total 150 points and that the tasks identified for those 40 user stories total 600 hours. We would then theorize that it is about 4 hours per story point?

Well, yes, the idea here is fine with two problems:

1. It’s important to remember that the relationship between points and hours is not an equivalence. It is not that 1 point equals 4 hours (which is what our example showed above). It is that 1 point equals a mean of four hours with a standard deviation of plus or minus say 45 minutes. This would mean that most of the time (68% of the time) the relationship would be that 1 point takes from 3:15 to 4:45 to finish. It would mean that almost always (98% of the time), one point would take between 2-1/2 and 5-1/2 hours to finish (two standard deviations).
2. The above approach assumes that 1-point stories and 13-point stories are estimated perfectly in relative terms. In other words, it assumes that if the mean duration of a one-point story is 4 hours, the mean of a 13-point story will be 13×4=52 hours. For many reasons, this is unlikely to be true. And the data I’ve collected from a variety of teams, shows that–as we’d expect–teams are not perfect, even though many are amazingly consistent.

So, what can we do to address these problems? A first really simple improvement is to calculate the average number of hours for stories of each size rather than one overall average. For example, in the example above we said that the 40 stories were 150 points in total and 600 hours for an average of 4 hours per point. But, if we averaged the 1-point stories on their own we might find that they were 3.2 hours per point, and the 2-point stories that were broken into tasks were 4.3 hours per point, and the 3-point stories were 4.1 hours, and so on.

We can then multiply that average number of hours by the number of stories on the product backlog of each size. An example using the average hours given above is shown in the following table:

First, notice in the above that the second column is hours per story, not hours per point. The two-point stories were assumed to take 4.3 hours per point so 8.6 (4.3×2) hours per story is shown in that column.

This table shows that we have five 1-point user stories. Each is expected to be about 3.2 hours so we expect to spend 16 hours total on the 1-point stories. Summing the last column in this table gives an expected total of 170.9 hours.

Note that this approach is subject to all the shortcomings that will have been introduced during the task identification and estimation step. Most importantly that the team will fail to identify all tasks. So this approach will estimate the number of hours to deliver the tasks identified. Some adjustment will need to be made to estimate the amount of work that the team failed to identify and the duration adjusted accordingly.

I’ll write about other improvements to this simple approach in upcoming blog posts.

In these situations a common question is whether the team should assign some number of story points to the work of fixing these legacy bugs.

If the team does not assign a story point value to this work, velocity will show the amount of “forward progress” the team is making each sprint. This has the advantage of making visible that the team is going more slowly through the work than it could if these legacy bugs had been fixed when originally found.

On the other hand, if the team does assign points to the bug-fixing effort, velocity comes to represent the team’s true capacity to accomplish work.

My usual recommendation is to assign points to the bug fixing. This really achieves the best of both worlds. We are able to see how much work the team is really able to accomplish but also able to look at the historical data and see how much went into the bug-fixing story each sprint. Knowing this can be helpful to a team and its product owner. For example, imagine a situation in which the product owner is considering suspending bug fixing for the next six sprints in order to add a valuable different feature into a release. If we know that the team’s full historical average velocity was 25 but that 5 points went to bug fixing each sprint, we know that suspending bug fixing for the next six sprints will result in 30 (6*5) more points of new functionality.

1. to be able to measure the amount of “business value” delivered to the organization, usually graphing this sprint by sprint
2. to be able to prioritize user stories by comparing the business value of each story to its cost

There are, unfortunately, some problems with these practices.

First, the value of small bits of functionality are often intertwined. When we get features that are too small (as good user stories are), it is very difficult to put a discrete value on each. Economists call this hedonic pricing. The classic example is putting values on the individual components of a house’s value—size, location, view, etc. How would you value each of those aspects of your house? You can see more on hedonic pricing on Investopedia and this example of pricing clothes dryers shows how complex hedonic pricing can get.

Second, the value of a small bit of functionality can often be said to be the total value of the product. For example, what is the value of the left front wheel of a car? Well, since I don’t want a car without that wheel, the value of it is the total value of the car.

Having identified two problems with assessing the “business value” of a user story, let’s look at a problem with determining the cost of the story.

The Problem of Shared Cost

Finally, when attempting to put business value points on small features (such as user stories) there is the additional problem of determining the cost of the feature. There is often a desire to do some form of return on investment (ROI) analysis on features by comparing the business value points to the cost in story points. However, with small features it is often the case that the true cost of implementing a feature is the sum of the cost of implementing multiple smaller parts of the system, some of which (e.g., architectural work) may not be separately estimated.

For example, the cost of refunding money to a credit card purchase should include some of the infrastructural work done to accept credit cards in the first place. That cost, however, was likely estimated as part of the story to “buy the items in my shopping cart.” Failure to share the cost of this credit card processing infrastructure between both the purchase and refund stories will lead to incorrect ROI decisions.

As with hedonic pricing, this is something economists have wrestled with for years. A simple example is a cow raised and sold for beef and leather. How should the costs of raising the cow be apportioned between the user stories of “beef” and “leather”? We could say the beef cost it all and we get the leather for free but that would be no more correct than the opposite. The benefits (beef and leather) need to be considered together in comparison to the cost of raising the cow.

What Do We Do?

So does this mean that we should never assign business value to features and that we should forego ROI analysis of user stories? Not necessarily. But this type of analysis is best used at the level of epics (large user stories) and themes (groups of stories) because value and cost can be appropriately assessed at those levels.

Project Success Sliders were initially created and devised by Rob Thomsett in his book, Radical Project Management. Sliders are a way for a product owner (or collectively all key stakeholders) to convey expectations to the team. By default there are six sliders, each of which reflects some dimension by which project success can be determined—for example, delivery of all planned features, quality, meeting an agreed upon schedule. This can be seen in this figure:

Each slider starts at a value of three along a continuum from one to five. The product owner or key stakeholders then moves sliders up or down to reflect the appropriate mix of factors in determining the success of the project. Stakeholders are prevented from simply moving all sliders to five by a rule that that every movement up must be offset by a corresponding move down.

If sliders are out of balance (e.g., too many have been moved to higher numbers), the green checkmark in the upper right is replaced by a red circle and number showing how far out of balance the sliders are as you can see in this figure:

Sliders are brought back into balance when the product owner or stakeholders made additional adjustments to the sliders as shown in this example:

Project success sliders can be a useful way to create an appropriate dialogue between stakeholders and team members about how success will be defined on the project.

Be sure to check out our Project Success Sliders tool and let me know what you think

First, a great deal of explicit risk management becomes unnecessary when a project uses an agile approach. The short iterations, single-minded focus on working software, heavy emphasis on automated tests, and frequent customer deliveries help teams avoid the biggest risk most projects face—that of eventually delivering nothing. So, it is not surprising that many agile projects forgo any form of explicit risk management. To put it in risk management terms, for these projects the likely savings from explicitly managing risks is outweighed by the cost of explicitly managing risk.

So while some short and inherently low-risk projects can be safely undertaken without any formal risk management, other projects are likely to benefit from explicitly managing risk. To see how we can do this, I want to describe a technique originally introduced by John Brothers in The Agile Times in 2004: the risk burndown chart.

Before we create a risk burndown chart we first need something akin to the typical risk census, which is merely a list of a project’s top risks along with the probability of the risk occurring and the size of the loss that would occur if it did. An example is shown in the following table, which I’ve simplified by eliminating a few columns that are not essential to this post:

Our simple risk census here describes each risk, provides an estimate of how likely the risk is to occur, the impact to the schedule if the risk did occur, and then the expected exposure to the risk which is the probability multiplied by the size of the loss. I recommend creating a risk census during the first sprint planning meeting and then updating it quickly during subsequent planning meetings as new risks are identified or as the probabilities or sizes of known risks change.

The risk burndown chart is then created by plotting the sum of the risk exposure values from the census. My recommendation is to sum only the top ten risks even if the team has identified more. Do this even if the top ten change over the course of the project. A sample risk burndown chart will look like this:

As with a regular release burndown chart, we should see a linear drop in risk over the course of the project, as shown in the red line of this next risk burndown chart.

When risk is not coming down at the appropriate rate, the team may want to budget some time in the next sprint to work directly on risk mitigation.

So as we can see from these examples, it is possible for risk management and agile project management to coexist on projects that need to explicitly do so. And the risk burndown chart provides a convenient way of visualizing changes in risk.

Teams that are new to agile project management often do not go far enough in eliminating these hand-offs. They often make the assumption that the programmers should finish programming a product backlog item before handing it off to the testers. This results in lengthy delays at the start of a sprint, when the testers are waiting for a first product backlog item to be handed to them. On a Scrum project, the unit of transfer between disciplines should be smaller than an individual product backlog item. That is, although there will always be some hand-offs (not everyone can be working on everything all the time), the amount of work being transferred from one person to the next should generally be as small as possible.

To facilitate these small transfers, Scrum teams learn to work by doing a little of everything all the time. Rather than an analysis phase (done without the programmer and tester) followed by a programming phase followed by a testing phase, a little of each of those activities is happening at all times.

Even with an emphasis on minimal hand-offs, there will be some product backlog items that will require a week or more of programming time before the programmer can give something even beginning to be testable to a tester. That’s OK. Not everything can be split as small as we might like. To avoid a flurry of testing activity toward the end-of-sprint, avoid bringing a bunch of these complex items into the same sprint. Instead of planning a sprint with, for example, three very large items that cannot be partially implemented, bring one or two large items into the sprint along with two or three smaller items. Some of the programmers can work on the large items, handing them over to testers whenever possible. The remaining programmers can work on the smaller items, ensuring a somewhat smoother flow of work to testers early in the sprint.

While your goal in each sprint should be to do a little bit of everything all the time, some backlog items are complex by nature and will not be completed until near or on the last day of a sprint. To avoid having multiple product backlog items wrapping up the last few days of the sprint, mix the size of the product backlog items you choose to bring into each sprint.