Fail Fast, Fail Often: An Interview with Victor Lombardi

Retrospectives are common. You’ve likely conducted one before. But how many companies are actually good at them? How many companies actually have the courage to be open and honest about their own shortcomings? My experience tells me that very few are. And that’s why Victor Lombardi’s recently released book, is so necessary: unlike the ones designers are used to seeing, Lombardi’s stories are full of objective, thoughtful, and insightful commentary.

An award-winning product designer, Victor Lombardi’s had a hand in over 40 different software and internet projects throughout the course of his career. And during that time he’s clearly paid attention to one thing: namely, all of the different ways in which a project can unfold. His new book, Why We Fail, tells over a dozen stories of projects gone awry.

So why do design projects fail? Many reasons. Lombardi attempts to answer the question from a number of angles: product ideation, design, development, and marketing. After reading his book, we brought additional questions to the discussion: How does bias factor in? Or branding? And, on a different level, what can we learn from healthcare.gov?

Our full interview appears, below. Additionally (as is always the case when we interview an author published by Rosenfeld Media) the publisher has graciously offered to give away a few books to readers. More information on that follows the interview!

Hey, Victor! Thanks for taking the time to chat. Throughout the book, you note a wide variety of places in which cognitive biases might affect an organization (“survivorship bias,” for example, is a perspective that exclusively favors success). Were you aware of bias and its effects from the outset or did you simply start to see bias the further you delved into your research?

I wasn’t expecting to hear about bias when I interviewed people for the book. Maybe that’s because I didn’t think people would open up this way. But they did.

I think it’s good therapy for us to talk through not only what we did during a project but also what we thought and felt. From there I brushed up on my psychology—Max Bazerman’s “Blind Spots” was particularly helpful—to explain the cognitive science behind the issues that led to failures.

Many companies find it (understandably) difficult to financially justify a culture that “embraces” failure. What advice do you have for them?

If senior management rules by ego, believing that the people at the top have the best ideas, then I’ve got nothing to say. They won’t hear my message.

For others, I think the overt message of “fail fast” is actually better framed as “experiment fast.” The most effective innovators succeed through experimentation. They’ve updated the traditional R&D department by stepping out of the lab and interacting directly with customers, running thoughtful experiments, and executing them quickly to learn quickly what works and what doesn’t.

Anyone doing user-centered design is already 80% of the way there. It makes a huge difference just to shift your process towards the scientific method, phrasing research questions as hypotheses and iteratively testing towards them. A key difference is in the results: instead of a lot of usability data to analyze and interpret, you get a true or false result. This makes it much easier to decide what to do next.

I recommend reading up on methods like customer development, lean startup, or by starting with the final chapter of my book.

In chapter four you recount the story of Wesabe and Mint, two startups who approached the financial space from slightly different perspectives. Wesabe suggested that users manually upload their financial data (in the name of privacy and security) whereas Mint.com automated this task (at the risk of perceived security). Both were minimum viable products, but one failed while the other succeeded. Can you speak a little as to what startups can learn, generally, from Wesabe and Mint.com’s subtle differentiation?

Wesabe was a useful service with a smart Web 2.0 strategy. Given more time and investment it would still be around. But certain classes of startups are dependent on attracting large numbers of customers in order to attract more investment. Mint.com chose features and designed them in a way that excelled at attracting customers. They won the competition even though Wesabe was superior in many ways.

But this isn’t true in every case. In the book I cover a broad spectrum of products: startups and mature products; mobile, web, and desktop software; hardware; and services. Different situations resulted in different lessons. I summarize the lessons at the end of each case study.

One of my favorite case studies in the book is Google Wave, in which you suggest that the first sign of trouble was that everyone had a different definition of what a “wave” actually was. Personally, I think this speaks to the strong connection between user experience, semantics and branding. How do we fail in this regard and how might we do better?

The UX field generally is not good at the conceptual design stage of creating new products compared to, say, industrial design or architecture. We fall in love with our first idea, and we can quickly and cheaply move from idea to working prototype—it isn’t natural to stay in the idea stage for a while to explore alternate solutions.

It’s unfortunate that Google Wave failed because the problem still exists. The solution was close. …maybe “Concept Design” should be my next book 😉

Chapter 7, titled “Do the right thing,” tells the story of Plaxo and Classmates.com, two companies who each decided to employ dark patterns to “better” their business. What other kinds of stories/examples did you consider including in this chapter that exhibited bad behavior?

In cases like Classmates.com I had no doubt the behavior was unethical. Others were less clear cut. Some of the things Plaxo did [ed: such as mass emailing its members’ contacts] that annoyed us back then are now accepted practice. So it’s relative. I decided against including others because there was no smoking gun, so I’ll refrain from mentioning them here as well. If you really want to know, you’ll have to buy me a drink sometime.

Last question! I know it’s a bit premature, but what, if anything, do you think designers might learn from the (highly publicized) failure of healthcare.gov?

Let’s say we solved for the myriad of political and vendor integration problems that plagued the project. What’s left are some intriguing customer experience issues. One seems to be that a long registration process is required before the customer can view prices of health plans, because the plans and prices are determined by your registration information. I don’t know how they ended up with that design, but the decision to design it this way sounds like a policy decision made around a conference table rather than through a design process that included running experiments.

What you can do if you find yourself in this situation is to acknowledge, out loud, that the goal of not showing prices prematurely is a good one, but the solution of making the customer do a lot of work up front is risky because more people will abandon the process before receiving any value from the site (see Wesabe vs. Mint). To mitigate this risk, you can generate alternate designs, mock them up, and test them out with customers.

Offhand, I can think of a few options:

• Let visitors browse plans upon arrival and show the range of prices next to each plan to give people a general idea of cost. Then show them the actual prices after registration.
• Show some realistic content so visitors know what factors will influence the price, like “Sally, a single mother of two in New York will pay $100/month for Plan X which includes benefits A, B, and C.”
• If just a bit of data is needed to determine price, like state and income, just ask for that data, and require registration later when people are ready to buy a plan.

Thanks, again, for taking the time, Victor! Your book was a pleasure to read.