Design Studios: The Good, the Bad, and the Science

Much has been written about the design studio methodology within the design community. In order to really understand how and why design studios work, though, designers must look beyond design—in particular, to social psychology and behavioral economics.

Design problems are tackled ad hoc far too often. Teams stumble into a design problem and chase the first idea they come up with. If the idea dead-ends, they may start over on a different solution.

In contrast, the design studio methodology offers a more structured approach. While variations have been proposed – from Ungar and White’s initial 2008 case study on the methodology to Will Evans’ more sophisticated variation described earlier this year – they all share the same basic flow.

At Case Commons, we use print-outs and post-its to share and critique ideas in our design studios.

Someone frames a design problem; they then assemble a team of designers, developers, and product managers; and, in turn, each team member individually brainstorms ideas. Members then share their ideas with their peers, who offer criticism and feedback. After additional rounds of brainstorming and group discussion, a smaller team takes the resulting ideas and produces a final design.

Why is this so much better than traditional approaches? Two things:

  • Design studio brings together a group with (theoretically) diverse and balanced skill sets (such as design, product management, and development).
  • Design studio prescribes an iterative, creative process (explanation, feedback, voting). Only surviving ideas are synthesized.

The Kennedy administration’s 1961 Bay of Pigs fiasco was one of many US foreign policy blunders attributed to “groupthink” and failure to develop alternatives.

Developing alternatives

In the typical ad hoc design approach, described above, a team starts with a single idea and begins to vet it. As it does so, the likelihood grows that the team will fail to discover flaws in the proposed solution, or discover better alternatives. If the team hits a dead-end, they may explore alternatives, but otherwise, confirmation bias pushes a team down it’s initial, possibly sub-optimal path. The risk, obviously, is that the team is ignoring other, better solutions that may be out there.

Design studio mitigates confirmation bias in two ways. First, it starts with an explicit brainstorming period, which gets lots of ideas out on the table, before people get wedded to any one idea. (One improvement on the design studio methodology would be to encourage and reward the most extreme ideas possible. Because of the fact that ideas begin to converge after the initial brainstorm, it is important to cast as wide an initial net as possible.) Second, design studio turns confirmation bias against itself. Confirmation causes individuals to fall in love with their initial ideas developed during brainstorming. This inoculates individuals against groupthink by giving them time to develop and fall in love with their own ideas before becoming exposed to other people’s ideas.

Design thinking has its own language for what is going on here: divergence (the creation of new alternatives) and convergence (the cross-pollination and merging of disparate ideas). For most groups, convergence occurs naturally; people are polite and avoid conflict, so compromise and consolidation of ideas happens easily. Divergence is, for many teams, less natural, and design studio are a great way of enforcing this crucial step.

Groups vs. individuals

What about the fact that design studios encourage a large group to tackle the design problem? We hear all the time that “two heads are better than one” and that diverse groups are better than uniform groups. Scientists in the 1960s developed a complementary task model that theorized that groups outperform individuals because they combine individuals with complementary talents and resources. This idea still informs our common sense today.

Research since the 1960s, however, has shown that it’s not that simple. Over the last century, social science has taught us that groups are subject to a whole range of dysfunctions such as conformity, groupthink, self-censorship, free-riding, and so forth. We’ve all experienced ineffective groups. The design community is particularly hostile to groups doing “design by committee.” Given all the problems with groups, when, if ever, groups are able to outperform individuals? Instead of using design studios, should we just have individuals do design?

Group vs. individual performance

In 1982, Hill published the most-cited paper (PDF) on group versus individual performance. Hill found that whether groups or individuals perform better depended on the type of task:

  • Brainstorming problems: When brainstorming, research shows that the best results come from having individuals brainstorm on their own and then pooling their ideas. The reasons groups perform worse at brainstorming include: production blocking (when one person talks, the others in the group are inhibited from being productive), social loafing (individuals don’t try as hard when they are judged as a group) and groupthink (individuals are less likely to pursue riskier, idiosyncratic leads in a group). In one experiment, researchers asked subjects to shout and clap as loudly as possible; they found that people clapped a remarkable 20-30% less loudly in a group than individually.
  • Creativity problems: When performing a creative task, groups outperform individuals—if all individuals are of similar skill level. However, the research shows that weak and even average members drag down group performance, and that this effect is stronger when the task is more challenging. For difficult creative problems, if a group contains both high- and low-performing individuals, any one of the high-performing individuals would probably out-perform the group on his or her own. For most creative problems, though, it’s safe to assume a group will generally perform as well or better than an individual.
  • Complex problems: Sometimes a group cannot even agree on what the problem is, or it may multiple overlapping sub-problems. For complex problems such as these, groups are about as good as their second-best member. Why? Complex problems usually don’t have intuitively obvious answers, so group members will have to convince one another. Whichever individual best understands the problem, then, will have to convince his peers. The group, then, will only be as good as the next-best individual, and whether he can grasp the solution put forth by the best individual. Since it’s usually hard to predict who would be the strongest individual for a given task, using a group makes sense.

These findings support the design studio methodology:

  • Identifying and framing a design problem is often a complex problem. This is outside the scope of design studio and done before setting it up. Research suggests it’s best done in a group setting.
  • The sketching phase of a design studio is a brainstorming problem, which research agrees should be performed individually.
  • The group sharing/feedback/discussion phase of a design studio is a creative problem: critiquing ideas, using proposed ideas to better understand the problem, searching for novel combinations of ideas, etc. Research agrees that it’s usually good to do this as a group. For harder problems or teams with widely varying skill levels, though, it may actually be better to have a single, highly skilled individual take the group’s ideas and design independently.

In fact, not only do social scientists support the design studio—social scientists essentially invented the design studio 50 years ago, except they called it Nominal Group Technique (NGT). NGT, though, only calls for one round of brainstorming and discussion, so technically the design studio is more akin to iterative-NGT.

What to Make of the Research

The overall morale of this story is that the design studio methodology is research-backed and theoretically sound. Compared to the usual ad hoc way designs often come together, teams would do well to incorporate design studios into their process. That said, there a few lessons to keep in mind as you apply it in your organizations:

The Kumar Model theorizes that design thinking involves a mix of research, analysis, synthesis and delivery. The design studio is a tool of the synthesis phase.

  • Design studio doesn’t prescribe how to analyze the domain or frame the design problem. Much design theory makes a distinction between analysis and synthesis. In that frame of reference, design studio is mostly about synthesis, and presupposes that the team has already done analysis (domain research, user research, etc) before the studio. This is not a fault of design studio, but it is worth noting as a constraint. Do your homework before the studio.
  • The research on group vs. individual performance gives two cautions about the group sharing/feedback/discussion phase of design studios. One, be careful who you invite: low-performing individuals drag down groups. Two, groups under-perform individuals for the toughest design problems; sometimes it’s best to go with a single high-performing individual.
  • NGT, upon which design studio is based, was originally a one-pass process. Research has noted over the years at how much this limits the cross-fertilization of ideas, to generate novel combinations of ideas. Design studio is at least two pass (design-discuss-design), but this still results in very little cross-fertilization. The more iterating in design studio, the better.

Hopefully this has shed some light on why design studios can work. What have your experiences been, though? Have you led or participated in them, and have you found them to be effective? I’d love to hear your real-life stories!

About the Author

Jim Lindstrom

Jim Lindstrom is a Senior Product Manager at Case Commons, Inc., where he helps lead the Casebook product team. He is passionate about scaling product teams and making UX design, agile development, and government 2.0 work together. He started his career working in product development at Motorola, where he developed wireless products for public safety clients. In 2008, he left to pursue an MBA at Columbia University and found a start-up that specialized in crowd-sourced market research. Follow him on Twitter!

Write for UX Booth

Contribute to UX Booth
Contribute

Contribute a guest post to UX Booth and let the community know what's important to you!



Comments

  1. Interesting reading, and I think parts of this apply outside of just the design industry. I’m a freelancer now, but prior to this I was in IT where we planned AV classrooms and often times it seemed like we could get no where with who thought what was the best way to get something done. And it tied right into not all of us where on the same performance level. A few people thought slack and half-assed was ok.

    The brainstorming one also makes total sense. And I’ve made use of a co-working group locally, where folks meet up to share their brainstorming to help narrow those results with great success.

    Anyways, interesting article.

  2. Mike – I love the idea of using co-working groups as an aid to brainstorming. Did you guys have sessions around a single idea, or rotate through some problems that each person brought? I’d love to hear about the process you used.

  3. There is nothing nicer than working with a group of highly talented individuals. When the ideas are flowing it can be quite exciting.

    In quite the opposite way it can be very frustrating when someone is holding you back in a brainstorming session.

    Nice article.

  4. How do you deal with the individual who is not performing as well as everyone else?

  5. Mark MacKay November 29, 2011

    What’s more interesting in development phases and after brainstorming and even after delivery is the detritus. Ideas that get clipped should be reconsidered during a post mortem – reviewing/reframing goals, process, hiccups, results.

    Be wary of labeling a different-thinker as slow or invalid or dead weight. Blame can sometimes infect the process and become the deliverable. And that’s not fun for anyone.

Related Posts