Share a Waterfallacy; Win a Book

January 11th, 2010

It seems like time for a new contest with the winner getting a free copy of Succeeding with Agile, my new book.

In Succeeding with Agile, I describe a waterfallacy as “a mistaken belief or idea about agile or Scrum created from working too long on waterfall projects.” And I give some examples, including these:

  • Scrum teams don’t plan, so we’re unable to make commitments to customers.
  • Scrum requires everyone to be a generalist.
  • Our team is spread around the world, and Scrum requires face-to-face communication.
  • Scrum is OK for simple websites, but our system is too complicated.

To enter the post, add a comment to this post telling us about one waterfallacy you’ve encountered and how you overcame that waterfallacy or convinced someone that it was a waterfallacy instead of the truth.

Let’s run this contest until midnight Mountain time on next Monday, 18 January. I’ll then announce winners on Tuesday, 19 January. I’ll pick two winners–the entry that I personally like the best plus one that will be randomly selected.

Good luck and let’s enjoying putting some waterfallacies to rest!

Posted in News, Transitioning to Agile | 38 Comments »

The Role of Leaders on a Self-Organizing Team

January 7th, 2010

Self-organization is a fundamental concept in agile software development. The Agile Manifesto includes the principle, “The best architectures, requirements, and designs emerge from self-organizing teams.” Yet a common misconception is that because of this reliance on self-organizing teams, there is little or no role for leaders of agile teams. Nothing could be further from the truth. In The Biology of Business, Philip Anderson refutes this mistaken assumption.

Self-organization does not mean that workers instead of managers engineer an organization design. It does not mean letting people do whatever they want to do. It means that management commits to guiding the evolution of behaviors that emerge from the interaction of independent agents instead of specifying in advance what effective behavior is.

Self-organizing teams are not free from management control. Management chooses for them what product to build or often chooses who will work on their project, but they are nonetheless self-organizing. Neither are they free from influence. Early references to Scrum were clear about this. In “The New New Product Development Game” from 1986, Takeuchi and Nonaka write that “subtle control is also consistent with the self-organizing character of project teams.”

A Scrum team’s job is to self-organize around the challenges, and within the boundaries and constraints, put in place by management. Management’s job is to come up with appropriate challenges and remove impediments to self-organization. That being said, the fewer constraints or controls put on a team, the better. If leaders overly constrain how a team solves the challenge given to it, self-organization will not occur. The team will shut down; because it has already been told so much about the challenge and how to solve it, it will wait to hear the rest. So how does an agile leader achieve the subtle balance between command and influence?

Suppose you are a ScrumMaster for a team. You’ve noticed that one team member, Jeff, is domineering and no one is willing to stand up to him. This team has self-organized—it has chosen to let Jeff make all key decisions. As the ScrumMaster for this team, though, you recognize that if Jeff continues to make all the decisions on his own it will impede the team’s efforts to improve. You consider having a private conversation with Jeff, but that is unlikely to change much. You contemplate stepping in and overruling some decisions he makes, but if you do it once the team will expect you to continue to do so, which won’t be good.

Then you begin thinking about subtle control and influence. Perhaps you decide to change the team’s dynamics by asking management to add someone new to the team, someone who is likely to stand up to Jeff. Or maybe you suggest to the enterprise architecture team that someone from its group attend key meetings—someone with the experience and background to challenge Jeff. No matter the specific problem, if you see that the team has self-organized in a way that impedes it, it is your responsibility to find a way to agitate, stir up, or otherwise disturb the status quo, so that the team adjusts, hopefully reorganizing in a more productive way.

There is more to leading a self-organizing team than buying pizza and getting out of the way. Leaders influence teams in subtle and indirect ways. It is impossible for a leader to accurately predict how a team will respond to a change, whether that change is a different team composition, new standards of performance, a vicarious selection system, or so on. Leaders do not have all the answers. What they do have is the ability to agitate teams (and the organization itself) toward becoming more agile.

For more details on how leaders can help teams and their organizations grow more agile, see Chapter 12 of Succeeding with Agile.

Posted in Scrum/Agile Roles | 19 Comments »

Remove Finish-to-Start Activities on Agile Projects

January 5th, 2010

A good ScrumMaster will continually nudge a team toward adopting improved technical practices that help them learn how to overlap their work. If a team doesn’t learn effective ways to do this, team members may settle on a less desirable approach: activity-specific sprints. An activity-specific sprint is as bad a practice as it would be an acronym. In this approach, the team decides to use one sprint for analysis and design, a second sprint for coding, and a third for testing. The team is split in thirds, with the analysts working one sprint ahead of the programmers and the testers working one sprint behind them.

This can be a very alluring approach. Not only does it seemingly solve the problem of how to overlap work but it also allows each type of specialist to work mostly with others of their own kind, which many may prefer until they become used to the close collaboration of a Scrum team. Unfortunately, the same disadvantages apply to activity-specific sprints as apply to activity-specific teams: too many hand-offs and a lack of whole-team responsibility.

One of the biggest problems with activity-specific sprints is that they create what are known as finish-to-start relationships. In a finish-to-start relationship, one task must finish before the next can start. For example, a Gantt chart on a sequential project may show that analysis must finish before coding can start and that coding must finish before testing can start. Good Scrum teams learn that this is not true; many activities can be overlapped. What is important is not when tasks start but when they finish. Coding cannot finish until analysis finishes and testing cannot finish until coding finishes. These are known as finish-to-finish relationships and are reinforced by Scrum’s sprint mechanism. All work is done at the end of the sprint, or it is returned to the product backlog.

With a little experience, most teams are able to see how to overlap some types of work and create finish-to-finish relationships between them. Teams easily find ways to overlap discussions of what users need and programming. They also soon find ways to overlap programming and testing. These activities lend themselves to iterative and incremental approaches: Get a few details from the users about what they need and then build a little of it; build a little and then test what you’ve built. Other activities, though, do not appear at first to be as amenable to an iterative, incremental approach. User experience design, database design, and architecture are often cited as work that needs to be done up front because the work must be viewed holistically. I argue that we should think holistically but work iteratively toward solutions. All sprint activities can and should overlap.

For specific guidance on how to overlap work in a sprint, see Chapter 14, “Sprints,” in Succeeding with Agile.

Posted in Sprinting | 21 Comments »

Mix the Sizes of the Product Backlog Items You Commit To

December 30th, 2009

Teams used to a sequential development process have become accustomed to hand-offs between specialists. Analysts hand their work to designers who hand it to programmers who pass it on to testers. Each of these hand-offs includes some overhead in the form of meetings, documents to read and perhaps sign, and so on. In part because of this overhead, the hand-offs tend to be of large amounts of functionality. In the purest meaning of a waterfall process, the entire application is handed from group to group.

Teams that are new to Scrum often do not go far enough in eliminating these hand-offs. They often make the assumption that the programmers should finish programming a product backlog item before handing it off to the testers. This results in lengthy delays at the start of a sprint, when the testers are waiting for a first product backlog item to be handed to them. On a Scrum project, the unit of transfer between disciplines should be smaller than an individual product backlog item. That is, although there will always be some hand-offs (not everyone can be working on everything all the time), the amount of work being transferred from one person to the next should generally be as small as possible.

To facilitate these small transfers, Scrum teams learn to work by doing a little of everything all the time. Rather than an analysis phase (done without the programmer and tester) followed by a programming phase followed by a testing phase, a little of each of those activities is happening at all times.

Even with an emphasis on minimal hand-offs, there will be some product backlog items that will require a week or more of programming time before the programmer can give something even beginning to be testable to a tester. That’s OK. Not everything can be split as small as we might like. To avoid a flurry of testing activity toward the end-of-sprint, avoid bringing a bunch of these complex items into the same sprint. Instead of planning a sprint with, for example, three very large items that cannot be partially implemented, bring one or two large items into the sprint along with two or three smaller items. Some of the programmers can work on the large items, handing them over to testers whenever possible. The remaining programmers can work on the smaller items, ensuring a somewhat smoother flow of work to testers early in the sprint.

While your goal in each sprint should be to do a little bit of everything all the time, some backlog items are complex by nature and will not be completed until near or on the last day of a sprint. To avoid having multiple product backlog items wrapping up the last few days of the sprint, mix the size of the product backlog items you choose to bring into each sprint.

Posted in Agile Planning | 11 Comments »

Build Trust Between Teams with Ambassadors

December 28th, 2009

On a distributed Scrum project, individual team members need to meet each other face to face. If the whole team cannot get together, one or two members from each team, at least, should spend time visiting team members in other cities. Think of them as ambassadors. I’ve found that the personal relationships established by ambassadors can be extremely valuable even long after the ambassador returns to native soil.

On one project I coached, for example, I had developers in Denver and Toronto. Teams in the two cities had been thrust together on a common project because of an acquisition, which initially had led to an unfriendly relationship between the two teams. Frank, a programmer in Denver, volunteered for a couple of two-week visits to Toronto. I knew the Toronto developers very well, having already worked with them for two years. I wanted to make sure we got the most benefit from Frank’s trips, so I talked with him about his hobbies and interests outside of work. When I discovered he was a rock climber, I contacted Marcel in Toronto, who was an obsessive climber. I asked Marcel to do me the favor of spending a little time with Frank, possibly setting him up with a guest pass to his indoor rock-climbing gym. Marcel very willingly did so, and the two of them became good friends and discovered they had other interests in common as well.

The budding friendship between Marcel and Frank served the project well right from the start. But it really paid dividends a few months later when a potential conflict started to emerge between departments on the periphery of our two-city project. The IT staff in Denver had named a server “Pandora” that would be used by the Toronto team. The Toronto team was furious over this and assumed the name had been intended as an insult because of the mythological story of Pandora’s box containing all the evils of mankind. I was in Toronto when the trouble started, so I asked Marcel to get Frank on the phone and to ask him if he would discreetly find out if the name had been intended as an insult. Two hours later Frank informed us that the employee who selected the name pulled it from a previously generated list of server names and had no idea who Pandora was. Because of the trust built between Marcel and Frank, we were able to quickly defuse the situation.

I write a great deal more about working with distributed teams in Chapter 18, “Distributed Teams,” in Succeeding with Agile.

Posted in Scaling Scrum/Agile | 4 Comments »

Synchronize Rather Than Overlap Sprints

December 23rd, 2009

On my first Scrum project, we started with only one team. That project soon grew to three teams, with the typical dependencies between them. I quickly arrived at what I thought would be a good way to manage those dependencies. I would stagger the sprint start dates by a week. The idea was that when a team went to start its sprint it would know the stories one of the other teams had recently committed to and which stories the other team was likely to finish.

Well, that part of my plan did work out well. But, overall, staggering the sprint start dates was a horrible idea. The biggest flaw in overlapping sprints is that there is never a time (except the end of the project) when all teams are done. One or more teams are always partway into a sprint. Some are planning a new sprint, others just planned a week ago, and still more teams will plan next week. This makes it difficult to give the full system to a customer for feedback or to an operations group for deployment.

A better idea is to have synchronized sprints, where all sprints end within a day or two of each other. Notice that all sprints do not necessarily need to end on exactly the same day. It is acceptable on a large project to have sprints that end over a two- or three-day period. In fact, there can be advantages to doing this. Allowing sprints to end over a two- or three-day period can make it easier for someone on multiple teams to attend all the review and planning meetings expected of someone on multiple teams. Additionally, it has the advantage in many cases of better accommodating remote team members who may travel into town for these meetings. A remote team member who is on multiple teams will find it easier to justify the travel time and expense if she can participate fully in each of her teams’ meetings.

Having synchronized sprints also does not mean that all teams must work in sprints of the same length. A project with multiple teams can accommodate different sprint lengths through the use of nested sprints. The most common use of nested sprints is when the various teams on the project cannot agree on a common sprint length, with some wanting two-week sprints and others wanting four-week sprints.

Don’t be tempted to overlap sprints. Instead, on multiteam projects, synchronize the sprints for maximum effectiveness.

More on sprint planning and scaling Scrum can be found in Succeeding with Agile.

Posted in Scaling Scrum/Agile | 14 Comments »

Reduce Manual Test Technical Debt

December 21st, 2009

When a team that has relied mostly or entirely on manual testing decides to adopt Scrum, it will quickly discover how hard it is to run short sprints when there’s a lot of manual testing to be done each sprint. It will also realize that unless it does something drastic, the technical debt will continue to accumulate. Teams in this situation can follow a three-step process to extricate themselves from at least the worst of these problems:

  1. Stop the bleeding.
  2. Stay current.
  3. Catch up.

The first priority of a team with technical debt in the form of an over-reliance on manual testing is to stop the bleeding, stop things from getting worse. The best tourniquet is to find ways to automate some of what is being tested manually. To mix metaphors, teams should find the low-hanging fruit: tests that will be easy to automate but save a lot of manual effort. Brian Marick, a leading authority on testing and coauthor of the Agile Manifesto, has told me that “the real low-hanging fruit is often not automating some test execution but automating other testing tasks, like populating databases or automatic navigation to the page where you’ll start manual testing. You’re not reducing the number of manual tests, but you’re reducing the total time it takes to run them.”

After the bleeding has been stopped, the situation will no longer be getting worse from sprint to sprint. Manual tests still will be added each sprint but the team is finding enough low-hanging fruit each sprint to offset the time needed to run the new manual tests. At this point, it is time to move to step two: learning to stay current. During this phase, the team focuses on learning how to write and automate tests for whatever new features are added during the sprint. While doing this, no more debt is accumulating, so the situation isn’t getting any worse, but it’s not yet getting any better either. Learning to add automated tests in the same sprint as the feature will be a new skill for the team. It won’t be as hard to learn as the initial skills were during the first phase but will require new discipline.

Eventually the team enters the final phase, which is when it catches up on additional outstanding testing debt. I generally tell teams that I coach that I don’t care how quickly the outstanding testing debt is brought down as long as it is indeed coming down. Obviously, I would prefer the debt to come down as fast as possible. But, by stating it this way I am emphasizing that what I am most concerned with are the first two steps: stop the bleeding and stay current.

For more on testing with an agile process like Scrum, pick up a copy of Succeeding with Agile.

Posted in Transitioning to Agile | 20 Comments »

The Forgotten Layer of the Test Automation Pyramid

December 17th, 2009

Even before the ascendancy of agile methodologies like Scrum, we knew we should automate our tests. But we didn’t. Automated tests were considered expensive to write and were often written months, or in some cases years, after a feature had been programmed. One reason teams found it difficult to write tests sooner was because they were automating at the wrong level. An effective test automation strategy calls for automating tests at three different levels, as shown in the figure below, which depicts the test automation pyramid.

Test Automation Pyramid

At the base of the test automation pyramid is unit testing. Unit testing should be the foundation of a solid test automation strategy and as such represents the largest part of the pyramid. Automated unit tests are wonderful because they give specific data to a programmer—there is a bug and it’s on line 47. Programmers have learned that the bug may really be on line 51 or 42, but it’s much nicer to have an automated unit test narrow it down than it is to have a tester say, “There’s a bug in how you’re retrieving member records from the database,” which might represent 1,000 or more lines of code. Also, because unit tests are usually written in the same language as the system, programmers are often most comfortable writing them.

Let’s skip for a moment the middle of the test automation pyramid and jump right to the top: the user interface level. Automated user interface testing is placed at the top of the test automation pyramid because we want to do as little of it as possible.

Suppose we wish to test a very simple calculator that allows a user to enter two integers, click either a multiply or divide button, and then see the result of that operation. To test this through the user interface, we would script a series of tests to drive the user interface, type the appropriate values into the fields, press the multiply or divide button, and then compare expected and actual values. Testing in this manner would certainly work but would be brittle, expensive, and time consuming.

Additionally, testing an application this way is partially redundant—think about how many times a suite of tests like this will test the user interface. Each test case will invoke the code that connects the multiply or divide button to the code in the guts of the application that does the math. Each test case will also test the code that displays results. And so on. Testing through the user interface like this is expensive and should be minimized. Although there are many test cases that need to be invoked, not all need to be run through the user interface.

And this is where the service layer of the test automation pyramid comes in. Although I refer to the middle layer of the test automation pyramid as the service layer, I am not restricting us to using only a service-oriented architecture. All applications are made up of various services. In the way I’m using it, a service is something the application does in response to some input or set of inputs. Our example calculator involves two services: multiply and divide.

Service-level testing is about testing the services of an application separately from its user interface. So instead of running a dozen or so multiplication test cases through the calculator’s user interface, we instead perform those tests at the service level.

Where many organizations have gone wrong in their test automation efforts over the years has been in ignoring this whole middle layer of service testing. Although automated unit testing is wonderful, it can cover only so much of an application’s testing needs. Without service-level testing to fill the gap between unit and user interface testing, all other testing ends up being performed through the user interface, resulting in tests that are expensive to run, expensive to write, and brittle.

For more on Scrum and agile testing, pick up a copy of Succeeding with Agile.

Posted in Sprinting, Transitioning to Agile | 17 Comments »

Make the Product Backlog DEEP

December 14th, 2009

Roman Pichler, author of Agile Product Management with Scrum: Creating Products That Customers Love, and I use the acronym DEEP to summarize key attributes of a good product backlog.

  • Detailed Appropriately. User stories on the product backlog that will be done soon need to be sufficiently well understood that they can be completed in the coming sprint. Stories that will not be developed for awhile should be described with less detail.
  • Estimated. The product backlog is more than a list of all work to be done; it is also a useful planning tool. Because items further down the backlog are not as well understood (yet), the estimates associated with them will be less precise than estimates given items at the top.
  • Emergent. A product backlog is not static. It will change over time. As more is learned, user stories on the product backlog will be added, removed, or reprioritized.
  • Prioritized. The product backlog should be sorted with the most valuable items at the top and the least valuable at the bottom. By always working in priority order, the team is able to maximize the value of the product or system being developed.

Product backlogs are discussed in much more detail in Succeeding with Agile.

Posted in Sprint Artifacts | 14 Comments »

The Fallacy of “One Throat to Choke”

December 10th, 2009

In speaking with some agile teams and consultants I occasionally hear two statements that I strongly disagree with. These statements are that “the product owner is the single wringable neck on the project” or that the “product owner is the one throat to choke.” These each mean the same thing, that the product owner is the person ultimately responsible for the success of the project.

This is wrong, however. On an agile project—as well as in many other cases—there is no single, wringable neck. To say there is a way of releasing the rest of the team from responsibility. And this is clearly wrong.
From a manager’s perspective it can be nice to always be able to point to one person and say, “That’s who I’ll blame if things go wrong.” But the “one throat to choke” argument is false. Historically, there may be one person who takes the blame for things when they go wrong, but that doesn’t mean that person was responsible for the failure.

Take the case of a sports team. At the start of a new season, who on a sports team do we say we’ll hold responsible for winning the championship? The coach? The owner? The star player? Teams that win championships find a way to win games, no matter the circumstances. If the game plan isn’t working, the coach and players adapt. If the star player is having a bad day, someone else steps up. The whole team feels responsible for winning somehow, some way. If the team loses, it may be tempting to blame one person or another, but the team knows that each one of them is accountable for the loss. It’s never just one person’s fault. In reality, there is no single, wringable neck.

Consider a nonsports analogy. If both parents were involved in raising a child (and assuming one of them isn’t abusive or obviously negligent), which parent is the one throat to choke if a child grows up to be a convicted felon? There is a reason we call it a parental unit. Raising a child is a team effort.

The only way to ever create an environment of shared ownership and responsibility is to let go of the notion of having one throat to choke. That doesn’t mean no one is responsible. That means that on a successful team, the team members must do their part, or even go beyond a perceived role, to ensure that the team reaches its goals.

For more on whole-team responsibility, see Succeeding with Agile.

Posted in Scrum/Agile Roles | 73 Comments »

« Older Entries
Newer Entries »