It’s summer and I’ve been thinking about sailing. I didn’t get to do any this summer, but I can still think about it. Thinking about sailing reminded me of the 1995 America’s Cup race between the US and New Zealand. That race is a great illustration of the importance of both getting close to our customers and of rapid feedback.
To design their boat, Team New Zealand made use of software that would allow them to simulate the impact of various design changes on the speed of the boat. They evaluated thousands of design decisions. Each day the simulations were run on a small network that was located a few feet from the dock. To further evaluate designs, Team New Zealand made two boats and each day would alter one with a design change to be evaluated. The two boats then raced each other to assess the impact of the design change.
By contrast, the U.S. boat had been designed and tested using massive supercomputers. But they were located hundreds of miles from the dock. This created significant feedback delays. Feedback was also slowed because the team had only a single boat on which to test changes.
Getting close to their customer and using rapid feedback cycles led to Team New Zealand winning the America’s Cup for the first time. If you are not cycling ideas past your customers quickly enough to get rapid feedback, consider moving closer to the dock.
First, the difference between a network next to the dock and one hundreds of miles away is a matter of milliseconds of latency. Is that really important enough to forgo supercomputer muscle?
Second, you’re talking about A/B testing. While I’m a fan, Jeff Atwood makes some entertaining points against it over at Coding Horror. His bottom line: “You can achieve a shallow local maximum with A/B testing — but you’ll never win hearts and minds.” In a competitive market a shallow local maximum can be the difference between failure and success, as in the ’95 America’s Cup. But to find a more general maximum you need to be ready to punctuate your equilibrium and take leaps of faith. An example of this would be the winged keel that won the ’83 America’s Cup. That’s not the kind of change you can implement overnight and test the next day.
So sure, rapidly cycle your ideas past your customer to find the local maximums. But don’t let that stop you, when the time is ripe, from going for a more general maximum by ignoring your customer’s feedback long enough to think different. *Then* A/B test the result.
Hi Alberto–
It’s been years so it’s good to see you here. Yes, there needs to be deeper thinking initially but at some point it becomes about refining a design rather than evaluating completely different alternatives. That’s where the A/B testing comes in, which is what you ultimately conclude with as well.
I don’t think latency was the issue–the key was the ability for ship designers, software developers, and the sailors to be able to work together immediately after a sail. For one to point to the boat and say “change that in the model” or to point to the model and say “change that in the boat.” This is hard when developers and their customers are not collocated.
Hi Mike. Good post. I think the difference between the computer next to the dock versus remote supercomputers is probably less about the milliseconds of network latency than the location of the teams using those systems. Where were the US design teams located relative to their boat and crew by chance? Also the millions of us down here that stayed up all night watching in 1983 would keenly point out that Ben Lexcen’s winged keel hung off a very non-American boat called Australia II.
Mike, a minor point – as a Kiwi I’d love this to be true;
“Getting close to their customer and using rapid feedback cycles led to Team New Zaland winning the America’s Cup–the first time a non-American boat had won.”
But the Australians won it in 1982, it went back to the US in 1987 and then NZ won it in 1995.
Hi Michael–
Unfortunately I don’t know exactly where the design team was located.
Hi Markus–
Whoops. I knew that. I wrote the blog on a plane and then posted it after landing. I’m not sure how I goofed since what spurred the post was listening to the Jimmy Buffett song, “Take It Back,” about the effort to win the Cup back. OK, now I really wish I were sailing…
Thanks, Markus. I’ve updated the post to say that it was the first time New Zealand won, which is what I’d meant.