We want to use agile when we are doing something that is new, or at least new to the team building it. If it’s something the team has done before over and over then the team probably doesn’t need an agile approach. To my mind, this is where some of the manufacturing analogies come in. If we are building the same car day after day, we learn pretty quickly all the nuances of building that car. We don’t need an agile approach because the novelty of the situation is low.

Novelty alone does not mean we should use an agile process. I went to my favorite Chinese restaurant for lunch today. I ordered an entree “triple extra spicy and with jalapenos.” It was probably the first time they cooked this particular dish that way and so it was a novel or unique order. The cook prepared it wonderfully though and because I could see into the kitchen I’m sure they didn’t need a daily standup or even TDD to make my lunch. (I might have noticed a kanban back there, however. So, in addition to novelty, the project needs a certain amount of complexity.

One final element I believe is required in making a project appropriate for agile is urgency. The timeboxes and iterations of an agile approach are devised to keep the intensity and focus going on a project. If there’s no urgency to the project, those are unneeded.

So let’s see how these three factors–urgency, complexity, and novelty–mix on various projects, starting of course with software projects. There couldn’t be a better fit. Software projects are notoriously complex. Each software project is largely a new endeavor. And in today’s world, there is almost always a sense of urgency.

But let’s look at one other situation where we commonly here about Scrum (in particular) being applied: getting married. At least a couple of times a year I hear about a couple who planned their wedding using Scrum. There is always a wedding backlog–buy cake, pick photographer, send invitations, pick dress, etc. How does planning a wedding do against the three factors I’m proposing. Sense of urgency? Check. There’s always a deadline and it’s usually pretty fixed. Complexity? Well, it’s not like a software project but it does have its own complexities often enhanced by non-functional requirements such as a fixed budget, who sits next to whom, the type of food to be served, the need to let Cousin Ira’s band perform at the reception, etc. Novelty? Yep. Most people don’t get married enough times with large celebrations that planning the event becomes second hand.

So, agile is most appropriate on any urgent project with significant complexity and novelty–and that includes software development and weddings. It does raise the question though of whether a couple’s first kiss at the end of the ceremony is a product backlog item or part of the done criteria for the overall product.

To design their boat, Team New Zealand made use of software that would allow them to simulate the impact of various design changes on the speed of the boat. They evaluated thousands of design decisions. Each day the simulations were run on a small network that was located a few feet from the dock. To further evaluate designs, Team New Zealand made two boats and each day would alter one with a design change to be evaluated. The two boats then raced each other to assess the impact of the design change.

By contrast, the U.S. boat had been designed and tested using massive supercomputers. But they were located hundreds of miles from the dock. This created significant feedback delays. Feedback was also slowed because the team had only a single boat on which to test changes.

Getting close to their customer and using rapid feedback cycles led to Team New Zealand winning the America’s Cup for the first time. If you are not cycling ideas past your customers quickly enough to get rapid feedback, consider moving closer to the dock.

This can be a very alluring approach. Not only does it seemingly solve the problem of how to overlap work but it also allows each type of specialist to work mostly with others of their own kind, which many may prefer until they become used to the close collaboration of a Scrum team. Unfortunately, the same disadvantages apply to activity-specific sprints as apply to activity-specific teams: too many hand-offs and a lack of whole-team responsibility.

One of the biggest problems with activity-specific sprints is that they create what are known as finish-to-start relationships. In a finish-to-start relationship, one task must finish before the next can start. For example, a Gantt chart on a sequential project may show that analysis must finish before coding can start and that coding must finish before testing can start. Good Scrum teams learn that this is not true; many activities can be overlapped. What is important is not when tasks start but when they finish. Coding cannot finish until analysis finishes and testing cannot finish until coding finishes. These are known as finish-to-finish relationships and are reinforced by Scrum’s sprint mechanism. All work is done at the end of the sprint, or it is returned to the product backlog.

With a little experience, most teams are able to see how to overlap some types of work and create finish-to-finish relationships between them. Teams easily find ways to overlap discussions of what users need and programming. They also soon find ways to overlap programming and testing. These activities lend themselves to iterative and incremental approaches: Get a few details from the users about what they need and then build a little of it; build a little and then test what you’ve built. Other activities, though, do not appear at first to be as amenable to an iterative, incremental approach. User experience design, database design, and architecture are often cited as work that needs to be done up front because the work must be viewed holistically. I argue that we should think holistically but work iteratively toward solutions. All sprint activities can and should overlap.

For specific guidance on how to overlap work in a sprint, see Chapter 14, “Sprints,” in Succeeding with Agile.

Remove Finish-to-Start Activities on Agile Projects by Mike Cohn is licensed under a Creative Commons Attribution-No Derivative Works 3.0 United States License.
Based on a work at Remove Finish-to-Start Activities on Agile Projects.

At the base of the test automation pyramid is unit testing. Unit testing should be the foundation of a solid test automation strategy and as such represents the largest part of the pyramid. Automated unit tests are wonderful because they give specific data to a programmer—there is a bug and it’s on line 47. Programmers have learned that the bug may really be on line 51 or 42, but it’s much nicer to have an automated unit test narrow it down than it is to have a tester say, “There’s a bug in how you’re retrieving member records from the database,” which might represent 1,000 or more lines of code. Also, because unit tests are usually written in the same language as the system, programmers are often most comfortable writing them.

Let’s skip for a moment the middle of the test automation pyramid and jump right to the top: the user interface level. Automated user interface testing is placed at the top of the test automation pyramid because we want to do as little of it as possible.

Suppose we wish to test a very simple calculator that allows a user to enter two integers, click either a multiply or divide button, and then see the result of that operation. To test this through the user interface, we would script a series of tests to drive the user interface, type the appropriate values into the fields, press the multiply or divide button, and then compare expected and actual values. Testing in this manner would certainly work but would be brittle, expensive, and time consuming.

Additionally, testing an application this way is partially redundant—think about how many times a suite of tests like this will test the user interface. Each test case will invoke the code that connects the multiply or divide button to the code in the guts of the application that does the math. Each test case will also test the code that displays results. And so on. Testing through the user interface like this is expensive and should be minimized. Although there are many test cases that need to be invoked, not all need to be run through the user interface.

And this is where the service layer of the test automation pyramid comes in. Although I refer to the middle layer of the test automation pyramid as the service layer, I am not restricting us to using only a service-oriented architecture. All applications are made up of various services. In the way I’m using it, a service is something the application does in response to some input or set of inputs. Our example calculator involves two services: multiply and divide.

Service-level testing is about testing the services of an application separately from its user interface. So instead of running a dozen or so multiplication test cases through the calculator’s user interface, we instead perform those tests at the service level.

Where many organizations have gone wrong in their test automation efforts over the years has been in ignoring this whole middle layer of service testing. Although automated unit testing is wonderful, it can cover only so much of an application’s testing needs. Without service-level testing to fill the gap between unit and user interface testing, all other testing ends up being performed through the user interface, resulting in tests that are expensive to run, expensive to write, and brittle.

For more on Scrum and agile testing, pick up a copy of Succeeding with Agile.

On more traditional, corporate projects, we see equivalent problems when teams organize around the layers of an application, including

• Reduced communication across the layers
• A feeling that design by contract is sufficient
• Ending sprints without a potentially shippable product increment

If structuring teams around the layers of an architecture is the wrong approach, what’s better? Rather than organizing around components, each team on a project can ideally be responsible for end-to-end delivery of working (tested) features. There are many advantages to organizing multiteam projects into feature teams:

• Feature teams are better able to evaluate the impact of design decisions. At the end of a sprint, a feature team will have built end-to-end functionality, traversing all levels of the technology stack of the application. This maximizes members’ learning about the product design decisions they made (Do users like the functionality as developed?) and about technical design decisions (How well did this implementation approach work for us?)
• Feature teams reduce waste created by hand-offs. Handing work from one group or individual to another is wasteful. In the case of a component team, there is the risk that too much or too little functionality will have been developed, that the wrong functionality has been developed, that some of the functionality is no longer needed, and so on.
• It ensures that the right people are talking. Because a feature team includes all skills needed to go from idea to running, tested feature, it ensures that the individuals with those skills communicate at least daily.
• Component teams create risk to the schedule. The work of a component team is valuable only after it has been integrated into the product by a feature team. The effort to integrate the component team’s work must be estimated by the feature team, whether it will occur in the same sprint during which it is developed (as is best) or in a later sprint. Estimating this type of effort is difficult because it requires the feature team to estimate the integration work without knowing the quality of the component.
• It keeps the focus on delivering features. It can be tempting for a team to fall back into its pre-Scrum habits. Organizing teams around the delivery of features, rather than around architectural elements or technologies, serves as a constant reminder of Scrum’s focus on delivering features in each sprint.

Of course, there will be occasions when creating a component team is still appropriate, for example when a new capability will be used by multiple teams or when the risk of multiple solutions being developed for the same problem is high. Overall, however, the vast majority of teams on a large project should be feature teams.

Additional advice on feature and component teams can be found in Chapter 10, “Team Structure,” of Succeeding with Agile.

This means that on a Scrum project, design is both intentional and emergent. The design emerges because there is no up-front design phase (even though there are design activities during all sprints). Design is intentional because product backlog items are deliberately chosen with an eye toward pushing the design in different directions at different times.

As an example of how the product backlog items can be sequenced to influence the architecture of the system, consider a workflow system I worked on. The system supported a fund-raising company that produced specialized T-shirts and similar products. School-age children would go door to door selling these items. The sales revenue would be split between the company and the organization the kids represented, such as a school, sports team, or other group. For each sale, the kid would complete a form and send it to the company, where it was scanned, sent through an optical character recognition (OCR) process, and converted into an order. To keep shipping costs down, orders from the same organization were batched together and sent back to the organization, after which the kids would hand-deliver the items.

Our software handled the entire process—from when the paper was received by the company until the shipment went out the door. Kids have notoriously bad writing and are bad spellers, so our system had to do more than just scan forms and prepare packing lists. There were various levels of validation depending upon how accurately we thought each order form had been read. Some forms were routed to human data-entry clerks who were presented the scanned form on one side of the screen, the system’s interpretation on the right, and an additional space to make corrections.

Because thousands of shirts were processed on the busiest days, this process needed to be as automated as possible. I worked with the product owner, Steve, to write the product backlog. After that I met with the development team to discuss which areas of the system were the highest risk or we were the most uncertain about how to develop. We decided that our first sprint would focus on getting a high-quality document to run through the system from end to end. It would be scanned, go through OCR, and generate a packing list. We would bypass optional steps such as deskewing crooked pages, despeckling pages, and so on but would prove that the workflow could be completed from start to finish. This wasn’t highly valuable but it was something that needed to be done, and it let the developers test out the general architecture. After we accomplished this, we had a basic database in place and could move documents from state to state, triggering the correct workflow steps.

Next the developers asked the product owner if they could work on the part of the system that would display a scanned document to a human who would be able to override the scanned and interpreted values. This was chosen as the second architectural goal of the project for three reasons:

• It was a manual step, making it different from the workflow steps handled already.
• Getting the user interface right was critical. With the volume of documents flowing through this system, saving seconds was important. We wanted to get early feedback from users to allow time to iterate on usability.
• After this feature was added, users could start processing shirt orders.

The project continued in this way for a few months and was ultimately tremendously successful, meeting all of the prerelease targets for reliability and throughput. A key to the success was that the product owner and technical personnel worked together to sequence the work. The closest the team got to a design phase was the first afternoon in the conference room when we identified risky areas and dark corners and decided which one we wanted to tackle first. From there the design emerged sprint by sprint, yet was intentionally guided by which product backlog items were selected to illuminate the dark corners and risks of the project.

Succeeding with Agile goes into much more detail on agile design, including how the roles of architect and user experience designer change with Scrum, the concept of emergent design, and how teams work together and with the product owner to deliver increments of functionality that guide the design of the final product.

But what about a bug found today in something the team wrote a month or a year ago? And what about the existing bug database that a team new to agile is almost sure to bring with them?

The ideal situation is to put the bugs right onto the product backlog. To see why, consider the situation from a user’s perspective: Do you think a user cares whether something is considered a New Feature or a Bug? No. The user just wants the system to behave in some way different from how it behaves today. They don’t care what it’s called.

But notice I called this the ideal situation. It isn’t always practical–and sometimes for reasons out of the team’s control or influence. Bug databases have a way of becoming embedded into organizations. The tech support group uses them as a primary way of communicating with the developers. The marketing group uses the bug database in a similar way. This makes it quit the bug database cold turkey.

In these cases, the most common solution teams use is to have two backlogs

• a product backlog of features
• a bug backlog

This approach is a bit harder on the team and the product owner but allows an agile team to work more easily with existing processes in the organization.

Sometimes when we make such accommodations to the overall organization, the accommodation can damage or destroy the agile adoption. Allowing everyone to work on five concurrent projects is an example of a crippling accommodation. Accommodating the organization’s use of a bug database is not crippling. It’s just a bit more work.

The product owner takes on most of the additional work by having to prioritize two separate work queues and then present them to the team in a more or less consolidated manner (“My top priorities are the first three items on the product backlog, then bugs #12403 and 12415, then these next two items on the product backlog…”). This isn’t too onerous as the items would need to be prioritized even if all were in one backlog.

So: Ideally bugs belong on the product backlog just like any feature request. But, that would often necessitate a significant change for the rest of the organization so two backlogs are used.

My first answer is always: Try to get the one person to move to where the rest of the team is. I don’t expect to see any moving trucks roll out when I ask this, but I have to ask. I figured if I keep asking, some team somewhere will eventually have the person move. Having one remote is a cost that must be borne by the full team. For the right person, it’s easily worth it. But sometimes, the person who is remote has not special skills, knowledge or experience to justify the added hassle.

So, assuming that we’ve tried the “move here” solution and it didn’t work, what can a team do that likes the physical task board but that has one remote member?

My recommendation is to continue to use the physical task board–it is simply too beneficial to the collocated team members to give it up in favor of a product backlog tool, especially if team members are already used to it and like it. How the team conveys information on the task board to the remote team member depends on what that person does.

Sometimes the remote person works remotely because they have a very specialized skill that couldn’t be filled in the office where the rest of the team is. This would be the case, for example, if our remote person is an expert in Windows internals and is writing boot-time code in C++. It would also be the case if our remote person has twenty years of experience in the domain and with some of our really old code. In these cases, the remote person usually (not always) works for a day or two relatively alone. The remote person in this case could identify during the daily Scrum what he or she will work on and not need further interaction with the task board until selecting the next task. Here, the remote person doesn’t really interact with the task board at all and interacts only with the team. Not ideal as I’d like the person to see the tasks, but this can work in some situations.

What is more common is for the ScrumMaster to take on the responsibility of updating an electronic version that mirrors the physical task board. A shared spreadsheet is normally sufficient for this, but some teams opt for a more specialized tool, which is fine. Many tools offer 5-10 user free licenses and since the tool is only needed by the ScrumMaster and one remote employees, the free license is adequate.

Normally the ScrumMaster updates the electronic task board once a day, usually right after the daily scrum meeting. Of course, reading the physical task board and updating the electronic one can be quite time-consuming because the ScrumMaster has to look at each task in both places to see if that item needs to be updated.

One good way of minimizing the time the ScrumMaster spends doing this is to mark the cards on the physical task board. The mark indicates “I’ve updated this task. Please update it in the online task board.” I like to use Post-It flags for this. As team members do their daily scrum, they stick one of these flags (of any color) on the card. If the estimate changes, if the task is done, if a new task is added, those cards are tagged with a flag. When the meeting is over, the ScrumMaster can very easily see which items need to be updated. The ScrumMaster removes the flags once the online task board is updated. This approach also works in situations where the ScrumMaster updates the board more than once a day. Any time someone changes the board, flag the task.

Other teams do something similar using color-coded dots. Anything touched on Monday is blue, Tuesday is red, and so on. Two problems occur though: First, the dot packs usually come with four colors to a pack so you have to buy a fifth color. Second, it’s a hassle to make sure we have the right color on hand.

• Big Visible Charts. Alistair Cockburn coined the term “Big Visible Charts” to describe the charts that agile teams like to hang on their walls. One of the most common of these is the sprint burndown chart, showing the number of hours remaining as of each day of the current sprint. Charts like these provide a strong visual reminder of the current state of the project. What is shown on these charts will get the attention of team members so display charts showing the most important information for that sprint. Ron Jeffries suggests considering big visible charts showing the number of passing customer acceptance tests, the pass/fail status of tests by day, sprint and release burndown charts, number of new stories introduced to the product backlog per sprint, and more.
• Additional feedback devices. In addition to big, visible charts, it is common for an agile team to use additional visual feedback devices in their workspace. One of the most common is a lava lamp that is turned on whenever the automated build is broken. I’ve also worked with teams that use flashing red traffic lights to indicate exceptional conditions such as an issue on a production server. Also popular are ambient orbs and Nabaztag rabbits, which are wireless programmable devices that can also be configured to change colors, speak messages, or wiggle their ears as a team desires. Devices like these make a workspace more dynamic, unobtrusively bringing into it information the team may find helpful.
• Everyone on your team. Each person on the team should ideally be able to see each other person on the team. This absolutely includes the ScrumMaster and ideally includes the product owner. I do understand, however, that product owners often have responsibilities to other groups outside the development team and so may sit near them instead. Still, in an ideal world the product owner would be visible to everyone in the team workspace.
• The sprint backlog. One of the best ways to ensure that everything necessary is completed in the sprint is to make the sprint backlog visible. The best way to do that is by displaying the sprint backlog on a wall, ideally in the form of a task board A task board is usually oriented in rows and columns with each row containing a particular user story and one index card or sticky note for each task involved in that story. Task cards are organized in columns, minimally including “To Do” “In Process,” and “Done.” In this way, team members are able to see work progressing across the task board during the sprint and all work to be done is visible at all times.
• The product backlog. One problem with running an endless series of sprints is that each can feel disconnected or isolated from the whole of a planned released or related set of new capabilities. A good way to reduce the impact of this problem is by displaying the product backlog somewhere clearly visible. This can be as simple as keeping the shoebox full of user stories written on index cards on a table in the middle of the team’s space. Even better, tack the index cards with those upcoming user stories on a wall where all can see them. This allows team members to see how the user stories they are working on in the current sprint relate to others that are coming soon.
• At least one big white board. Every team needs at least one big whiteboard. Locating this in the team’s common workspace encourages spontaneous meetings. One developer may start using the board to think through a problem; others may notice and offer to help.
• Someplace quiet and private. As important as open communication is, there are times when someone needs some peace and quiet. Sometimes this is for something as simple as a private phone call. Other times it can be to think through a particularly challenging problem without being interrupted.
• Food and drink. It’s always a good idea to have food and drink available. These don’t need to be fancy, and they don’t even need to be provided by the organization. I’ve worked with plenty of teams that buy a small under-desk refrigerator and share the expense of buying water bottles or soda for it. Other teams buy a coffee machine, depending on team preferences. Some teams rotate bringing in snacks, both healthful and not.
• A window. Windows are often a scarce commodity and are doled out to an organization’s favored employees. One of the nice things about an open workspace is that windows are shared. Even if the view is only of our parking lot and can only be seen across three messy desks, at least I can see the window and some natural light.

It’s unlikely that every one of these will be visible from your workspace, but the more of them visible, the better. Let me know what else you think should be visible from within the ideal agile workspace.

In general, a team should feel comfortable working on multiple product backlog items at the same time during an iteration. A typical seven person team will plan between five and ten items into an iteration. They’ll usually be closer to the low end of that range with a one- or two-week iteration and closer to the high end with a four-week iteration. Perhaps surprisingly, though, iteration length has less influence on the number of product backlog items selected than you might think. It seems that teams with longer sprints simply have larger product backlog items.

A seven-person team will rarely be at its most efficient when all team members swarm onto a single product backlog item. There’s too much opportunity for them to get in each other’s way for this to be a good long-term strategy. However, an entire team swarming onto a single product backlog item can be a very effective learning technique and one that I often encourage. If you are part of a team that hasn’t yet learned how all disciplines can work well together, limiting the entire team to one product backlog item in progress at a time is something you might want to try. This forces people to quickly learn ways to work in small batches so that work can, for example, be transferred from programming to testing in multiple tiny increments.

Similarly, if you are on a team where each developer grabs a product backlog item and works independently on it throughout an iteration, a rule of “only one item in progress at a time” is a good way to experience the benefits of working in smaller batches.

So, while I think swarming in this way is an excellent technique to use sometimes, I don’t think it should be the default way of working for very many teams. Some may benefit from it over the long term, but most will find that it introduces too many opportunities to be in someone else’s way as they try to make progress. I consider it equivalent to a drill that a team may do to improve a skill–good to use for practice but not the way to do something forever.