Interview with the Authors of Remote Research

Nate Bolt and Tony Tulathimutte, of the creative research firm Bolt | Peters, have conducted literally hundreds of interviews; so it's only fitting that we return the favor. Read on to learn more about what design research means to you and how you and yours, and how you can get started. If that's not enough, we're even giving away books!

Research is what Bolt | Peters does best. As a digital research firm, their job is to uncover and determine the ways in which real users use real products with their “natural” environments. In order to achieve the most accurate data possible, researchers at Bolt | Peters employ a broad variety of research methods. One method in particular, known as Remote Research, has garnered considerable attention.

The book cover for Remote Research, published by Rosenfeld Media.

In their forthcoming new book, authors Nate Bolt and Tony Tulathimutte address the questions they receive while practicing Remote Research. From the mundane, “what is remote research,” to the more intriguing “how does it compare with more conventional design research practices?” their book promises to cover it all.

In an odd turn of events, we at the UX Booth found ourselves with the opportunity to sit down with these esteemed researchers and give them a dose of their own medicine, asking questions of our own. In the following interview, learn what design research means to Nate and Tony and even how you can get started. If that’s not enough incentive to employ research within your own design practice, we’re even giving away some copies of the soon-to-be-released book. Enjoy!

I would hazard a guess that most of our readers who are interested in experience design (and remote research), aren’t in a position or at a company where it’s treated as a first-class citizen. Although the question is frequently posed, I have to ask: how do you suggest these people introduce remote research into their product-development process?

Add a couple of informal UX research sessions into your next design project, whether or not design research is specifically included in the budget or timeline. Don’t ask anyone if they think it’s a good idea, just do it. It only takes 2-3 hours, and costs nothing. All the standard remote UX tools have free trials, and you can grab a screen sharing tool like GoToMeeting or Acrobat Connect, a screen recording program like Quicktime 10 or iShowUHD, and you’re good to go. Then edit the recording in iMovie or another free video editor. You can also use an in-person tool like Silverback.

When you show photos, user quotes, or quick video highlights to your stakeholders, it will blow them away if they weren’t expecting it, and it can all be done in roughly the amount of time we each spend on Facebook and Twitter every day.

Can you briefly explain the idea behind time-aware research and “live recruiting?”

In traditional user research, you ask someone in advance to come in to a boring usability lab and pretend to care about the interface or design you’re researching. With time-aware research, you intercept them live with a web tool like ethnio or a mobile-based form, and ask them to participate in a phone interview just as they’re about to use an interface, on their own initiative and timeline. What’s important is that you’re capturing someone right at the moment when they care about what they’re doing, whether it’s making a donation, buying a phone, purchasing enterprise software, etc. The criteria by which they make click decisions are totally different when they really need to book travel that minute.

A great example of a time-aware research project is the popular Track Your Happiness project at Harvard, which uses SMS messages to intercept people and answer questions about their happiness into a mobile form. The happiness data is reported in “real-time,” which provides far more accurate data than people filling out some survey about how happy they were last month. Basically, it eliminates our misrepresentations about interactions by peeking into real-life behavior and time.

Do you find that you obtain better results presenting/observing reactions to early sketches or wireframes vs. functional prototypes or fully-functioning applications/products?

The answer to this question is actually a funny comment on the current state of the web. We’ve tested 2,300 participants over ten years, with at least 40% of those sessions involving a prototype, a scanned napkin sketch, static JPGs, or wireframes. We never use the word “prototype,” and almost always, after we redirect people to a static wireframe, they say “yeah, this looks like other sites I’ve used.” In other words, even with a crap sketch, as long as it clicks through to another crap sketch, people can relate that to their normal browsing experience. That’s how crappy parts of the web are. People aren’t shocked by a black and white version where only one thing works.

That said, the more functional interactions our participants have, the more insight we get, so the results do get incrementally better with more fidelity, but testing sketches is still incredibly useful.

In today’s web application space, two terms are hot topics: “agile development,” and “user experience design.” How do you see research (in general) keeping step in a fast-paced, agile development environment? Applications that are in-development simply don’t have the users with which to conduct time-aware research, so I wanted to ask for your thoughts on how prudent experience/interaction designers can proceed in this “pressure-cooker” situation. (I’m aware of—and eagerly anticipate reading—Anders Ramsay’s book on Agile Experience Design)

Go guerilla. Andy Budd from ClearLeft loves this approach and so do we. The philosophy is: instead of forgoing testing, or trying to get it approved, just grab someone whose paycheck doesn’t involve your design or development, and watch them interact. The same goes for remote testing—you don’t even have to leave your chair to observe someone’s functional experience with an interface.

We did a recent study for Mint.com, who don’t practice Agile but do release very rapidly, and they made changes on the fly based on what the designers and developers were seeing in remote testing. We conducted it from a conference room where it was easy for their team to come observe, minimizing the gap between developers and designers and “the research.” They just saw UX issues and addressed them.

Research is seen primarily as an academic pursuit: how did you get started and how do you suggest others do the same?

Everyone here at Bolt | Peters studied some variation of Cognitive Science, Linguistics, or Human-Computer Interaction. I worked as a design intern pushing pixels for a web development company in 1998 called Clear Ink, and I convinced them to let me pitch usability to their clients, since everyone was listening to punk 21 yo. kids back then. Eventually, one of the clients thought it sounded like a good idea and our first study yielded designs that got a 150% conversion rate increase and millions of dollars more in web sales. Then it was off and running.

So the steps I’d suggest are:

  • Take a class or two in a related field.
  • Intern somewhere that conducts UX and IxD.
  • Just do research without asking.
  • Measure results and highlight the awesomeness.

That’s all, folks!

Thanks again to Nate and Tony for taking the time to answer our questions and share their thoughts with the community. Remote Research is an exciting, emerging methodology which increasingly appeals to contemporary design endeavors.

From what I’ve read thus far, it’s going to be an awesome book. So good, in fact, I would recommend purchasing two copies: one to keep and one to share—which, come to think of it …

Want to win a copy?

In case you hadn’t noticed, Rosenfeld Media is one of the primary sponsors of UX Booth this quarter. As a consequence, we’ve got 3 copies of Remote Research to give away to our community. To enter, simply leave a comment below and make sure to include your twitter ID. Next, be sure to follow us (@UXBooth) on twitter. If you’re selected, we’ll be in touch via a Twitter DM to obtain your mailing address. Thanks, and good luck!

Rather just purchase one?

Readers of UX Booth are entitled to a 15% discount off any of the fine products found at Rosenfeld Media. Simply enter code UXBOOTH at checkout.

About the Authors

Nate Bolt

Nate is fascinated by the personal, social, and cultural role of technology, and how research and design can transform those roles. After pioneering and directing the User Experience department at Clear Ink in 1999, which included the construction of Natural Environment and Remote Observation laboratories, Nate co-founded Bolt | Peters. He now serves as el presidente, where he has overseen hundreds of user research studies for Sony, Oracle, HP, Greenpeace, Electronic Arts, and others. Beginning in 2003, he led the creation of the first moderated remote user research software, Ethnio, which is being used around the world to recruit hundreds of thousands of live participants for research.

Tony Tulathimutte

Tony specializes in user research for video games, working for Time Warner’s GameTap service and NHN USA’s online games portal, ijji.com. (His honors thesis was about video game controllers, and his Master’s thesis concerned game menu interfaces; in this way, he makes the world a better place.) He joined Bolt | Peters in 2007, and has done remote studies for Autodesk, AAA, HP, and Harvard Business Review; more recently, he was the lead researcher on the player experience study for EA’s Spore.

Additional Resources

About the Author

Andrew Maier

Andrew is a lifelong student of the design community, who co-founded the design publication UX Booth in 2008 to share his journey. He currently serves as its Editor-in-Chief. When he's not heading user-centered design initiatives for clients, Andrew dabbles in civic design. He lives in Seattle's Capitol Hill neighborhood.

Related Articles

45 Comments

  • Adam Bitner Reply

    I’d love a copy of Remote Research! We do remote usability testing at my company and would certainly like to read tips from the gurus.

    Twitter ID: @apbitner

  • Jake Reply

    Sounds like a great book, sign me up for a copy @suimple. For remote research I use TeamViewer to run the meeting and record and edit with Camtasia.

  • James Costa Reply

    Awesome interview, Andrew! Definitely want my hands on a copy. Username on Twitter is jamescosta :-)

  • Bruce Floyd Reply

    Great interview! I saw Nate speak at SxSW and was looking forward to this particular book. In fact, I have several books from Rosenfeld Media and would recommend them all, especially “Web Form Design” by Luke Wroblewski.

  • Bruce Floyd Reply

    Damn, forgot to include my twitter acct:

    @brucefloyd

    Thanks!

  • Yael Reply

    Very interesting.. great taster! @yaellevey

  • Fahed Reply

    Will not play the “I’ve red all the article” guy. I’m reading it, but can’t miss my chance to be given a copy… So what is remote research ? I will find ou in a moment i hope…

  • Aaron Irizarry Reply

    Hey Andrew thanks for posting this interview. I have been eyeballing this book for a bit now.

    I just started a new position at a larger company and research is a big part of it… I would love a copy of this book and could put it to some serious use.

    So when you are dishin’ out the love don’t forget to dish some to @aaroni268 :)

  • marina Reply

    Hey, thanks for sharing the interview and I would love to have a copy of the book, here is my Twitter username: @simpleasthat_

  • Dante Murphy Reply

    Nice interview. I really enjoyed Nate’s presentation at Ix10. We do a lot of remote research, and even with all of our experience there were some great new tools and pointers.

    Would love to get a copy of the book for our departmental library…and one for myself.

    @dantemurphy

  • Sarah Keogh Reply

    We perform remote testing all the time, and I agree, we need to look further into more catch and release tactics. I hope I win the book but I think I will have to ask if we can buy it in our team, Thanks for the article!

  • Jerome Reply

    I’m feeling lucky. Sorry everyone!

    Twitter: @jeromebot

  • lsirivong Reply

    Interesting interview, and it sounds like a great book. I especially appreciate the advice to “just do it” as someone who “isn’t treated as a first-class citizen” ha.

  • Calvin C. Chan Reply

    Inspiring interview, especially “Intern somewhere that conducts UX and IX research”. I’m working on it. Definitely adding this book to my read list. Even better if I can win a copy from UXBooth :-)

    @calvincchan

  • Falka Reply

    Interesting interview, good points there. I’ve been trying to do more usability testing at my company lately and doing it without asking others for permission really seems to be working. Hopefully people will start asking for it at some point.

    And because the book looks interesting and I’ve come here mainly to try my luck, here it is: @falkowata

  • Travis Smith Reply

    Thanks for preaching the gospel of Usability and Usability testing! I can’t wait to read the new book.

    @travisbsmith

  • Marko Prljić Reply

    You’re giving away a book? Sure, I’m very interested in reading this.

    • Andrew Maier Reply

      Hey Marko, if you wanna win, you’ve gotta enter! Leave us your twitter ID and follow @uxbooth. Thanks for your interest.

  • Jeremy Reply

    More testing! More Books!

    @introworks

  • Blair Keen Reply

    I would love a copy of your book.

    Twitter: @AdditveUK

  • Coleman Reply

    Nice! Looking forward to reading it!

    @colemandash

  • darine Reply

    Thanks guys for the great opportunity to win this amazing book

    twitter.com/sdarine

  • Ducky Reply

    You guys really have helped me get into UX – it’s recently become a major aspect of my job and I am thankful for your constant updates.

    3 cheers!

    @duckyxxx

  • Ravi Reply

    Sounds really thoughtful !!! Should be the second book of UX I would read after Kuniavsky. Notify to my twitter ID @rvkmrj for a book copy

  • Randy Hammons Reply

    Very cool – would love to win that book! cheers! @optikinescant

  • Susan Reply

    We do a fair amount of user research, but very little of it is remote. Would love to get my hands on a copy of this book!

    @sachopra

  • Břeťa Passinger Reply

    I got from Rosenfeld Media @ books (in PDF) – Web Form Design by Luke Wroblewski and Mental Models by Indy Young.
    Both worth the money spent.

    And I do dare to tweet about that @BretaPassinger

  • Martin Pownall Reply

    Really interesting interview. Would love to get a copy of the book.

    @mspownall

  • Emma Pickering Reply

    Absolutely agree with the focus on “real users use real products with their “natural” environments”.

    If you want womething to work in the ‘real world’ then assess and design it in the real world.

    tw: @Emma_Pickering

  • Imad Mouaddine Reply

    Thanks for the interesting blog and for the book :) tw: @imadmouaddine

  • Tony Collen Reply

    This book will be a great resource — not only are our customers and users remote to us, but I also telecommute.

    – @tcollen

  • Mr Bootle Reply

    Great article and I would love to have a read of the book. It would be very useful for my work as a few of my clients are remotely located.

    @profb

  • Drew Reply

    @drewshapter Certainly sounds like one that needs to be read

  • Lorraine Paterson Reply

    I’d love a copy of this book. Own a couple of Rosenfeld books and they are always insightful reads.

    Twitter ID: @lorraine_p

  • Patrick Lajeunesse Reply

    I’ve gotten so much value from guerilla testing… but I’m currently prepping for a bunch of remote testing sessions. So good timing. ;)

    @patlaj on twitter.

  • Matthew Kammerer Reply

    The winners, chosen by Random.org, are: @aaroni268 @apbitner @lorraine_p.

    Congratulations!

  • Aaron Irizarry Reply

    Thanks Gang! I am super stoked for this book… cant wait to put it to some good use.

  • Jessica Reply

    @JessMarieHolt – very interesting article. Always the debate to go remote or not.

  • Gavin Wray Reply

    Just read this post and realised I’m too late to ask for the free copy! It would have been timely – ‘Remote Research’ is the first title we’re reading at the new Birmingham UX Book Club (UK):

    http://uxbookclub.org/doku.php?id=birmingham

    Looks like a great title. Useful insight too in this post on just doing it and where to find the tools to start.

    • Matthew Kammerer Reply

      It will be a great book to cover in the book club! Be sure to use the discount code UXBOOTH :).

  • Gavin Wray Reply

    Thanks for the discount code Matthew. Much appreciated!

Leave a Comment on This Article