Archive for the ‘Sprint Artifacts’ Category

Make the Product Backlog DEEP

Monday, December 14th, 2009

Roman Pichler, author of Agile Product Management with Scrum: Creating Products That Customers Love, and I use the acronym DEEP to summarize key attributes of a good product backlog.

  • Detailed Appropriately. User stories on the product backlog that will be done soon need to be sufficiently well understood that they can be completed in the coming sprint. Stories that will not be developed for awhile should be described with less detail.
  • Estimated. The product backlog is more than a list of all work to be done; it is also a useful planning tool. Because items further down the backlog are not as well understood (yet), the estimates associated with them will be less precise than estimates given items at the top.
  • Emergent. A product backlog is not static. It will change over time. As more is learned, user stories on the product backlog will be added, removed, or reprioritized.
  • Prioritized. The product backlog should be sorted with the most valuable items at the top and the least valuable at the bottom. By always working in priority order, the team is able to maximize the value of the product or system being developed.

Product backlogs are discussed in much more detail in Succeeding with Agile.

Posted in Sprint Artifacts | 14 Comments »

Bugs on the Product Backlog

Monday, July 6th, 2009

A common question is whether bugs belong on the product backlog. Before I address that question, let me clarify that the question refers to bugs that are unrelated to functionality being coded during the sprint. If someone finds a bug during a sprint that is related to the features being worked on, the best thing to do is yell, “Hey, Mike, the boojum code is broken when I…” The second best thing to do is to stick a new task card on the task board saying “Fix the boojum code; it breaks when…”

But what about a bug found today in something the team wrote a month or a year ago? And what about the existing bug database that a team new to agile is almost sure to bring with them?

The ideal situation is to put the bugs right onto the product backlog. To see why, consider the situation from a user’s perspective: Do you think a user cares whether something is considered a New Feature or a Bug? No. The user just wants the system to behave in some way different from how it behaves today. They don’t care what it’s called.

But notice I called this the ideal situation. It isn’t always practical–and sometimes for reasons out of the team’s control or influence. Bug databases have a way of becoming embedded into organizations. The tech support group uses them as a primary way of communicating with the developers. The marketing group uses the bug database in a similar way. This makes it quit the bug database cold turkey.

In these cases, the most common solution teams use is to have two backlogs

  • a product backlog of features
  • a bug backlog

This approach is a bit harder on the team and the product owner but allows an agile team to work more easily with existing processes in the organization.

Sometimes when we make such accommodations to the overall organization, the accommodation can damage or destroy the agile adoption. Allowing everyone to work on five concurrent projects is an example of a crippling accommodation. Accommodating the organization’s use of a bug database is not crippling. It’s just a bit more work.

The product owner takes on most of the additional work by having to prioritize two separate work queues and then present them to the team in a more or less consolidated manner (“My top priorities are the first three items on the product backlog, then bugs #12403 and 12415, then these next two items on the product backlog…”). This isn’t too onerous as the items would need to be prioritized even if all were in one backlog.

So: Ideally bugs belong on the product backlog just like any feature request. But, that would often necessitate a significant change for the rest of the organization so two backlogs are used.

Posted in Sprint Artifacts, Sprinting | 24 Comments »

Using a Task Board with One Remote Team Member

Sunday, May 31st, 2009

I want to address a question I was emailed yesterday but that I receive frequently. What should we do in the case when the team really likes having a physical task board, but when one team member is remote?

My first answer is always: Try to get the one person to move to where the rest of the team is. I don’t expect to see any moving trucks roll out when I ask this, but I have to ask. I figured if I keep asking, some team somewhere will eventually have the person move. Having one remote is a cost that must be borne by the full team. For the right person, it’s easily worth it. But sometimes, the person who is remote has not special skills, knowledge or experience to justify the added hassle.

So, assuming that we’ve tried the “move here” solution and it didn’t work, what can a team do that likes the physical task board but that has one remote member?

My recommendation is to continue to use the physical task board–it is simply too beneficial to the collocated team members to give it up in favor of a product backlog tool, especially if team members are already used to it and like it. How the team conveys information on the task board to the remote team member depends on what that person does.

Sometimes the remote person works remotely because they have a very specialized skill that couldn’t be filled in the office where the rest of the team is. This would be the case, for example, if our remote person is an expert in Windows internals and is writing boot-time code in C++. It would also be the case if our remote person has twenty years of experience in the domain and with some of our really old code. In these cases, the remote person usually (not always) works for a day or two relatively alone. The remote person in this case could identify during the daily Scrum what he or she will work on and not need further interaction with the task board until selecting the next task. Here, the remote person doesn’t really interact with the task board at all and interacts only with the team. Not ideal as I’d like the person to see the tasks, but this can work in some situations.

What is more common is for the ScrumMaster to take on the responsibility of updating an electronic version that mirrors the physical task board. A shared spreadsheet is normally sufficient for this, but some teams opt for a more specialized tool, which is fine. Many tools offer 5-10 user free licenses and since the tool is only needed by the ScrumMaster and one remote employees, the free license is adequate.

Normally the ScrumMaster updates the electronic task board once a day, usually right after the daily scrum meeting. Of course, reading the physical task board and updating the electronic one can be quite time-consuming because the ScrumMaster has to look at each task in both places to see if that item needs to be updated.

One good way of minimizing the time the ScrumMaster spends doing this is to mark the cards on the physical task board. The mark indicates “I’ve updated this task. Please update it in the online task board.” I like to use Post-It flags for this. As team members do their daily scrum, they stick one of these flags (of any color) on the card. If the estimate changes, if the task is done, if a new task is added, those cards are tagged with a flag. When the meeting is over, the ScrumMaster can very easily see which items need to be updated. The ScrumMaster removes the flags once the online task board is updated. This approach also works in situations where the ScrumMaster updates the board more than once a day. Any time someone changes the board, flag the task.

Other teams do something similar using color-coded dots. Anything touched on Monday is blue, Tuesday is red, and so on. Two problems occur though: First, the dot packs usually come with four colors to a pack so you have to buy a fifth color. Second, it’s a hassle to make sure we have the right color on hand.

Posted in Sprint Artifacts, Sprinting | 29 Comments »

Why There Should Not Be a “Release Backlog”

Sunday, February 8th, 2009

I haven’t heard the term “release backlog” in many months, but it’s come up in three conversations over the past week. So, I want to share my thoughts on whether a team should have a Release Backlog in addition to the conventional Product and Sprint (or Iteration) Backlogs.

First, let’s clarify what people mean when they refer to a “release backlog.” A release backlog is a subset of the product backlog that is planned to be delivered in the coming release, typically a three- to six-month horizon. A release backlog would presumably contain the same type of items as on a product backlog. My preference for this, of course, would be user stories. So, at the start of some release planning horizon, a product owner with help from the team and other stakeholders would example the product backlog, select some amount of high priority work and move those items to the release backlog. Later, in sprint (iteration) planning, the team would examine the release product and select some amount of work to do. (Notice how this already goes against existing agile literature that says the team selects top priority items for the sprint from the product backlog?)

I am opposed to the use of a release backlog for a couple of reasons:

First, in all projects that are not being done as fixed-scope contracts (and, arguably, even then) we don’t know exactly what user stories or features will be delivered in a release. To say that there is a release backlog may not imply a guarantee that all features on it are delivered. However, it does create additional work as items are moved off back and forth between the product and release backlogs. This will happen as the team’s velocity changes (even by small amounts) from sprint to sprint. If the release backlog is to show what a team will deliver in a release, it should contain an amount of work equal to the number of story points (or ideal days) times the number of remaining sprints. If you have an average velocity of 20 and 6 sprints left, the release backlog should contain 120 points worth of work. Suppose the team finishes 24 points of work in a sprint. The backlog is now down to 96 points (120-24) but should contain 100 (5 sprints left x an average velocity of 20). Things get even more difficult if that good velocity of 24 increases the team’s average velocity to 21. In that case the release backlog should contain 5×21=105 points of work; we need to move 9 points of work from the product backlog to the sprint backlog. If there’s a release backlog, this moving of items back and forth from product backlog to release backlog will happen every sprint.

Second, introducing a release backlog creates additional confusion. We’ve already overloaded the word “backlog” with “product backlog” and “sprint backlog.” Why confuse things further with a third type of backlog unless there is an immensely compelling reason to do so?

Third, introducing the concept of a Release Backlog actually makes it harder for product owners to do one of the most important things they should be doing–looking at the features or user stories that just miss the cut of being in a release. When I look at a product backlog and count down the number of story points that I anticipate will be in the release, the most interesting thing to me is not necessarily the set of user stories that will make it into the release. I’m often just as interested in the next few user stories below that cut-off point. That is, I want to know what user stories won’t quite make it into the release. After all, we say that one of the benefits of agile (on some, not all, projects) is the ability to decide later if we should release a few sprints early (if a lot has been done), on the planned date, or perhaps a sprint or two late if we want to add more functionality before a release. This is made more difficult with a Release Backlog because a product owner now needs to examine both a Product Backlog and a Release Backlog to make these decisions.

So, what do I propose instead?

I advocate that all teams track their velocities. This leads to a graph like this:

Velocity as graphed over the last nine sprints.

This graph shows a team calculating three average velocities:

  1. Long term average, defined as the mean of the last eight sprints
  2. Worst case average, defined as the mean of the worst three chosen among the last eight sprints
  3. Best case average, defined as the mean of the best three chosen among the last eight sprints

You can choose your own ways of calculating long-term, best-case, and worst-case average velocities. These are just the ones I like. We use these average velocities as shown in this figure:

Using velocity to predict how much will be done by a given date.

In this figure, the product backlog is shown on the left as a stack of index cards. We use the three average velocities multiplied by the number of remaining sprints to draw arrows pointing into the product backlog. The range created by these arrows indicates the likely amount of work finished by the planned date. Our release plan is the work defined by the arrows. Rather than a Release Backlog as a new work product or artifact, my equivalent of a release backlog is the product backlog and these arrows pointing into it.

As you can imagine, predicting velocity as a range is much more likely to be accurate than using a single point estimate (“Our velocity is 31.”) as has been implied by everyone I’ve heard talk about a Release Backlog. Additionally, looking at figures like these should make it apparent that the amount of work expected to be delivered in a release will change from sprint to sprint. Pointing arrows into a product backlog is much easier and more helpful than creating a new work product, the Release Backlog.

Posted in Agile Planning, Sprint Artifacts | 48 Comments »

Non-functional Requirements as User Stories

Friday, November 21st, 2008

A common challenge with writing user stories is how to handle a product’s non-functional requirements. These are requirements that are not about specific functionality (“As a user of a word processor, I want to insert a table into my document.”), but are rather about an attribute or characteristic of the system. Examples include reliability, availability, portability, scalability, usability, maintainability. As you can see from that list, non-functional requirements are often referred to as “-ilities”. Of course, not all non-functional requirements end in “-ility.” We also have security, performance, robustness and so on.

Thinking back into the dark ages, I can remember when I first read about “non-functional requirements.” The term threw me for a loop. “If it’s non-functional, why do I care about it?” I’m sure the author of that book clarified this for me a page later, but the term has always seemed an odd one to me. I prefer to think of non-functional requirements as “constraints” we put on the system. When a product owner says “this system must perform adequately with 100,000 concurrent users” the product owner is putting a constraint on the development team. The product owner is effectively saying, “Develop this software any way you’d like as long as you achieve 100,000 concurrent users.” Each constraint we put on a system narrows the design choices a bit; calling these “constraints” rather than “non-functional requirements” helps us remember this.

Fortunately constraints / non-functional requirements can be easily handled as user stories. Here are a couple of examples:

  • As a customer, I want to be able to run your product on all versions of Windows from Windows 95 on.
  • As the CTO, I want the system to use our existing orders database rather than create a new one sot that we don’t have one more database to maintain.
  • As a user, I want the site to be available 99.999% of the time I try to access it so that I don’t get frustrated and find another site to use.
  • As someone who speaks a Latin-based language, I might want to run your software someday.
  • As a user, I want the driving directions to be the best 90% of the time and reasonable 99% of the time.

As you can see from these I was quite easily able to stick with the “As a <type of user>, I want <some goal>, so that <some reason>” template that I prefer for most user stories. I do this for a couple of reasons but want to comment further on only one of them here. Consider the example of the CTO constraining the team to use the existing orders database. (This was the real situation; the team was considering a second orders database that would be sychronized at night. The CTO overheard this and said “no!”) If we wrote this requirement as simply “Must use existing orders database” it would be entirely possible that a few months down the road we wouldn’t remember why we should be constrained in that way. We’d ask the product owner if she cared if we used a secondary database, and she’d say she had no objections. And we’d make a mistake of using one. Embedding the person who wants something can be very useful.

But, you should be careful not to get obsessed with that template. It’s a thinking tool only. Trying to put a constraint into this template is a good exercise as it helps make sure you understand who wants what and why. If you end up with a confusingly worded statement, drop the template. If you can’t find a way to word the constraint, just write the constraint in whatever way feels natural

Posted in Agile Planning, Sprint Artifacts | 20 Comments »

Visualizing a Large Product Backlog With a Treemap

Wednesday, July 2nd, 2008

In the early days we promoted agile only for small teams because that was where it originated. We had plenty of experience to say that agile worked well on seven- to ten-person teams. We were also quick to learn the techniques that allowed agile to scale up to around 20-40 people.

These days, though, there are many truly large projects being done with agile. In preparing for this posting I started counting on my fingers the number of 100+ person projects I’m aware of. I quickly ran out of fingers. I’ve been involved in a couple of 500+ person projects and am aware of three projects that each have over 1,000 people. We are truly at the point of doing large scale agile.

Unfortunately, the charts and tools we use for such large projects have not entirely caught up with us. For example, the time-tested Scrum burndown chart is absolutely wonderful but is really limited to showing only one thing: a single team’s rate of progress through their product backlog. This month I want to write about visualizing large product backlogs using a technique I’ve been advocating with my clients for three years but will be writing about here for the first time.

Suppose you have a very large product backlog–such as one with lots of themes (groups of related user stories) or being worked on by multiple agile teams. The best way to visualize this product backlog is with a treemap. Treemaps were invented in the 1990s by Dr. Ben Shneiderman as a way of visualizing hierarchical (that is, tree-structured) data.

The following is a simple treemap of a two-story house:

treemap of two floors of a house

In this treemap, each rectangle is sized to represent the relative area of the room. From it you can see that the master bedroom is about twice the size of either kid’s room and a little larger than the downstairs family room. The combined green area of the downstairs rooms is slightly larger than the area of blue upstairs rooms. From this very simple treemap you can get a feel of certain aspects of this house.

To visualize a product backlog with a treemap we need to conceptualize it as hierarchical data. We can do this a couple of different ways. For example,

Rental Car Theme

  • As a traveler, I can rent a car
  • As a traveler, I can get collision insurance on a car rental policy.
  • As a traveler, I can request a baby car seat be inside my car.

Airplane Theme

  • As a traveler, I can request an aisle seat.
  • As a frequent flyer, I can request an upgrade to first class.

Or we could create a product backlog as a tree with levels for

  • Team
    • Product Backlog Items Being Worked On By That Team

The following figure shows a product backlog as a treemap. There are five themes in the treemap. (Theme 4 is in the top left; Theme 1 is in the bottom right. You can click on it to enlarge it but be sure t come back.) Each theme is made up of a number of individual user stories. Story 4-28 (indicating the 28th story of theme 4) is in the top left. I’ve used this “theme-dash-story” notation for simplicity only for this example. I wouldn’t do that on a real project, of course.


a treemap

The size of each story in the preceding figure represents the number of story points that the story was assigned when estimated. Colors can be used on a treemap to represent an additional attribute of the data. On agile projects I’ve used colors to represent whether a user story has been developed or not. One color coding we used was:

  1. Done
  2. Started but not done
  3. Not started but not planned to have been started yet
  4. Not started but it should have been started by now
  5. Blocked

I’ve also used color to indicate which team would work on a user story / product backlog item or whether the item was ready for development or not. Treemaps are very flexible in this regard. Check out the link to the stock market as a treemap at the end to see a great example of using color.

A good treemap is interactive–you can mouse over, click on regions of it, and so on. For example, in the treemap above you’ll notice that some of the user stories of Theme 4 are so small you can’t read them. Clicking on a part of Theme 4 should zoom the treemap in so it displays only Theme 4, meaning more room is available and more detail can be shown as below:

Zoomed in on Theme 4

Sometimes zooming isn’t enough. A good treemap implementation will also display additional detail when mousing over part of it as shown in this example:

A treemap with an active popup

Treemaps are an excellent way of visualizing large product backlogs. To date, I’ve made use of various open source or relatively inexpensive general treemapping tools to create these on my projects or for my clients. Hopefully now some of the leading agile tool vendors begin to incorporate this type of visualization technique into their products. I would love to be able to visualize a large product backlog and interact with directly from within some of those tools. Until then, check out some of the links below for useful ways to create treemaps. The ones in this posting were created with IBM’s free Many Eyes tool.

Useful links:

  1. The stock market as a treemap
  2. IBM’s Free Many Eyes tool
  3. A good interactive example of a treemap
  4. Read news headlines as a treemap
  5. A simple JavaScript implementation you could add to your project homepage

Posted in Scaling Scrum/Agile, Sprint Artifacts | 15 Comments »

Advantages of the “As a user, I want” user story template

Saturday, April 26th, 2008

In my user stories book and in all my training and conference sessions on user stories I advocate writing user stories in the form of “As a <type of user>, I want <some goal> so that <some reason>.” While I consider the so-that clause optional, I really like this template.

At a recent conference, someone asked me why. Because I get that question fairly often I want to give three reasons why here:

Reason 1
Something significant and I’m tempted to say magical happens when requirements are put in the first person. Obviously by saying “As a such-and-such, I want….” you can see how the person’s mind goes instantly to imagining he or she is a such-and-such. As for the magic, Paul McCartney was interviewed and asked about why the Beatles songs were so amazingly popular. One of his responses was that their songs were among the first to use a lot of pronouns. Think about it: She Loves You, I Wanna Hold Your Hand, I Saw Her Standing There, I Am The Walrus, Baby You Can Drive My Car, etc. His point was that these helped people more closely identify with the songs. I tried briefly to find a source for this interview tonight and the closest I found was this. The information in that reference fits my recollection of hearing McCartney says this during a radio interview in 1973 or 74 that I assume was recording when the Beatles were together.

Reason Two
Having a structure to the stories actually helps the product owner prioritize. If the product backlog is a jumble of things like Fix exception handing, Let users make reservations, Users want to see photos, Show room size options, and so on, the product owner has to work harder to understand what the feature is, who benefits from it, and what the value of it is.

Reason Three
I’ve heard an argument that writing stories with this template actually suppresses the information content of the story because there is so much boilerplate in the text. If you find that true then correct it in how you present the story. I’ve seen backlogs in Word that present the boilerplate in grayed text with the unique parts in black. I’ve created product backlogs in Excel that use column headings to filter out the common text. Look and you’ll see what I mean:

sample user stories in Excel

Seriously take a look at the image above before continuing….I’ll wait……Please, really look at it before reading further…..

OK, here’s how I bet you read the spreadsheet. I bet that as you read most of the rows you added in the “As a…”, “I want….” and “so that….” text, possibly even by looking at the heading as you read across each row. I’ve experimented by asking people and this is what most people do. If that text is unnecessary, why do we mentally mouth the words to ourselves or even glance at the heading while reading the row? Perhaps it’s not so non-essential after all.

For now, and probably a long time to come, I’ll be sticking with the “As a <type of user>, I want <some goal> so that <some reason>” template for these and other reasons.

Posted in Sprint Artifacts | 65 Comments »

Writing User Stories for Back-end Systems

Monday, February 25th, 2008

I was asked recently how to go about writing user stories for a back-end financial system. This is an interesting example and is a question I get asked a lot, so I thought I should answer it here. This example brings up a couple of key interesting challenges:

  • While there may be users of the system they are not often not direct users (i.e., with hands on the keyboard waiting for something to happen)
  • The functionality is often larger than will fit in one iteration

So that we can deal with these challenges, let’s make up some more context for the example. Suppose our financial system takes in a lot of daily data and produces flat files that will be sent to banks and other partners at the end of the day. Some of the files are simple formats. Other files are in more involved formats with multiple record types within the file and possibly with multiple lines for some of the transactions. This would be a fairly standard batch-processing type application.

I want to write user stories in the template I recommend in the book: As a , I want so that . So, let’s deal with these challenges in order and first try to figure out who the user will be in our stories? Given the context provided above the user, is probably a bank or business partner. This will let us write stories like “As a bank, I want…” It’s entirely possible that we will want to get more specific and sometimes write stories for more specific users:

  • As a commercial bank, I want….
  • As a savings & loan, I want…
  • As the Bank of America, I want… (assuming we have some specific business partnership that provides BofA with unique functionality)

Now, what is it these banks want?

At a high level we can start out with some epics, perhaps like these:

  • As a bank, I want to receive a file showing all checks to be cleared so that I can debit and credit the right accounts.
  • As a bank, I want to receive a correctly formatted 5300 file so that I can adjust balances as appropriate.
  • As a bank, I want to receive a file showing the status of all resubmitted transactions so that I can update the database with new information.

(Note that I am liberally making up specific business rules or details. Presumably someone writing the real user stories could add real and relevant detail instead.)

First, notice it is OK to humanize the bank with “As a bank, I want…” Programmers do this all the time in conversation. “OK, suppose I’m the bank and you send me a 5300 file with a bad record. In that case, I’ll…”

As I said at the start of this post, one of the challenges in an application like this is that some of these user stories we just made up could be large. Suppose our mythical file format 5300 includes a header, three different record types, and a footer. We could break up the 5300 file user story above into things like:

  • As a bank, I want to receive a 5300 file with the correct header and footer so that I can process the file correctly.
  • As a bank, I want to receive a 5300 file with correctly formatted prearranged payment records so that I can process those correctly.
  • As a bank, I want to receive a 5300 file with correctly formatted deposit entry records so that I can process them.
  • As a bank, I want to receive a 5300 file with correctly formatted addendum records so that I can process them.

If those were still too big, I would find ways to split them smaller based on the specific formatting of the records. Suppose our deposit entry record type sometimes fits on one line but can also span multiple lines. In that case I could write these user stories:

  • As a bank, I want to receive a 5300 file with correctly formatted single-line deposit entry records so that I can process them.
  • As a bank, I want to receive a 5300 file with correctly formatted deposit entry records that include continuation lines so that I can process them.

Is the system “potentially shippable” after each of these user stories is delivered? Yes, it kind of is. While I’ll admit that we’re unlikely to find a bank that will take a 5300 file with only a header and a footer, it’s possible. Especially if we think of the situation in which the bank is simultaneously writing their own software to process the 5300 files we send them. Imagine meeting with the bank before the start of the next iteration and perhaps agreeing, “OK, in this iteration we’ll create some blank 5300 files with just headers and footers. But that will let us make sure that the communication problems are worked out, that you can read the files we write, and so on….” These stories also work well because they will likely fit with how the programmers will want to build the system.

Posted in Sprint Artifacts | 23 Comments »