Interview with the Authors of Remote Research

Research is what Bolt | Peters does best. As a digital research firm, their job is to uncover and determine the ways in which real users use real products with their “natural” environments. In order to achieve the most accurate data possible, researchers at Bolt | Peters employ a broad variety of research methods. One method in particular, known as Remote Research, has garnered considerable attention.

The book cover for Remote Research, published by Rosenfeld Media.

In their forthcoming new book, authors Nate Bolt and Tony Tulathimutte address the questions they receive while practicing Remote Research. From the mundane, “what is remote research,” to the more intriguing “how does it compare with more conventional design research practices?” their book promises to cover it all.

In an odd turn of events, we at the UX Booth found ourselves with the opportunity to sit down with these esteemed researchers and give them a dose of their own medicine, asking questions of our own. In the following interview, learn what design research means to Nate and Tony and even how you can get started. If that’s not enough incentive to employ research within your own design practice, we’re even giving away some copies of the soon-to-be-released book. Enjoy!

I would hazard a guess that most of our readers who are interested in experience design (and remote research), aren’t in a position or at a company where it’s treated as a first-class citizen. Although the question is frequently posed, I have to ask: how do you suggest these people introduce remote research into their product-development process?

Add a couple of informal UX research sessions into your next design project, whether or not design research is specifically included in the budget or timeline. Don’t ask anyone if they think it’s a good idea, just do it. It only takes 2-3 hours, and costs nothing. All the standard remote UX tools have free trials, and you can grab a screen sharing tool like GoToMeeting or Acrobat Connect, a screen recording program like Quicktime 10 or iShowUHD, and you’re good to go. Then edit the recording in iMovie or another free video editor. You can also use an in-person tool like Silverback.

When you show photos, user quotes, or quick video highlights to your stakeholders, it will blow them away if they weren’t expecting it, and it can all be done in roughly the amount of time we each spend on Facebook and Twitter every day.

Can you briefly explain the idea behind time-aware research and “live recruiting?”

In traditional user research, you ask someone in advance to come in to a boring usability lab and pretend to care about the interface or design you’re researching. With time-aware research, you intercept them live with a web tool like ethnio or a mobile-based form, and ask them to participate in a phone interview just as they’re about to use an interface, on their own initiative and timeline. What’s important is that you’re capturing someone right at the moment when they care about what they’re doing, whether it’s making a donation, buying a phone, purchasing enterprise software, etc. The criteria by which they make click decisions are totally different when they really need to book travel that minute.

A great example of a time-aware research project is the popular Track Your Happiness project at Harvard, which uses SMS messages to intercept people and answer questions about their happiness into a mobile form. The happiness data is reported in “real-time,” which provides far more accurate data than people filling out some survey about how happy they were last month. Basically, it eliminates our misrepresentations about interactions by peeking into real-life behavior and time.

Do you find that you obtain better results presenting/observing reactions to early sketches or wireframes vs. functional prototypes or fully-functioning applications/products?

The answer to this question is actually a funny comment on the current state of the web. We’ve tested 2,300 participants over ten years, with at least 40% of those sessions involving a prototype, a scanned napkin sketch, static JPGs, or wireframes. We never use the word “prototype,” and almost always, after we redirect people to a static wireframe, they say “yeah, this looks like other sites I’ve used.” In other words, even with a crap sketch, as long as it clicks through to another crap sketch, people can relate that to their normal browsing experience. That’s how crappy parts of the web are. People aren’t shocked by a black and white version where only one thing works.

That said, the more functional interactions our participants have, the more insight we get, so the results do get incrementally better with more fidelity, but testing sketches is still incredibly useful.

In today’s web application space, two terms are hot topics: “agile development,” and “user experience design.” How do you see research (in general) keeping step in a fast-paced, agile development environment? Applications that are in-development simply don’t have the users with which to conduct time-aware research, so I wanted to ask for your thoughts on how prudent experience/interaction designers can proceed in this “pressure-cooker” situation. (I’m aware of—and eagerly anticipate reading—Anders Ramsay’s book on Agile Experience Design)

Go guerilla. Andy Budd from ClearLeft loves this approach and so do we. The philosophy is: instead of forgoing testing, or trying to get it approved, just grab someone whose paycheck doesn’t involve your design or development, and watch them interact. The same goes for remote testing—you don’t even have to leave your chair to observe someone’s functional experience with an interface.

We did a recent study for Mint.com, who don’t practice Agile but do release very rapidly, and they made changes on the fly based on what the designers and developers were seeing in remote testing. We conducted it from a conference room where it was easy for their team to come observe, minimizing the gap between developers and designers and “the research.” They just saw UX issues and addressed them.

Research is seen primarily as an academic pursuit: how did you get started and how do you suggest others do the same?

Everyone here at Bolt | Peters studied some variation of Cognitive Science, Linguistics, or Human-Computer Interaction. I worked as a design intern pushing pixels for a web development company in 1998 called Clear Ink, and I convinced them to let me pitch usability to their clients, since everyone was listening to punk 21 yo. kids back then. Eventually, one of the clients thought it sounded like a good idea and our first study yielded designs that got a 150% conversion rate increase and millions of dollars more in web sales. Then it was off and running.

So the steps I’d suggest are:

  • Take a class or two in a related field.
  • Intern somewhere that conducts UX and IxD.
  • Just do research without asking.
  • Measure results and highlight the awesomeness.

That’s all, folks!

Thanks again to Nate and Tony for taking the time to answer our questions and share their thoughts with the community. Remote Research is an exciting, emerging methodology which increasingly appeals to contemporary design endeavors.

From what I’ve read thus far, it’s going to be an awesome book. So good, in fact, I would recommend purchasing two copies: one to keep and one to share—which, come to think of it …

Want to win a copy?

In case you hadn’t noticed, Rosenfeld Media is one of the primary sponsors of UX Booth this quarter. As a consequence, we’ve got 3 copies of Remote Research to give away to our community. To enter, simply leave a comment below and make sure to include your twitter ID. Next, be sure to follow us (@UXBooth) on twitter. If you’re selected, we’ll be in touch via a Twitter DM to obtain your mailing address. Thanks, and good luck!

Rather just purchase one?

Readers of UX Booth are entitled to a 15% discount off any of the fine products found at Rosenfeld Media. Simply enter code UXBOOTH at checkout.

About the Authors

Nate Bolt

Nate is fascinated by the personal, social, and cultural role of technology, and how research and design can transform those roles. After pioneering and directing the User Experience department at Clear Ink in 1999, which included the construction of Natural Environment and Remote Observation laboratories, Nate co-founded Bolt | Peters. He now serves as el presidente, where he has overseen hundreds of user research studies for Sony, Oracle, HP, Greenpeace, Electronic Arts, and others. Beginning in 2003, he led the creation of the first moderated remote user research software, Ethnio, which is being used around the world to recruit hundreds of thousands of live participants for research.

Tony Tulathimutte

Tony specializes in user research for video games, working for Time Warner’s GameTap service and NHN USA’s online games portal, ijji.com. (His honors thesis was about video game controllers, and his Master’s thesis concerned game menu interfaces; in this way, he makes the world a better place.) He joined Bolt | Peters in 2007, and has done remote studies for Autodesk, AAA, HP, and Harvard Business Review; more recently, he was the lead researcher on the player experience study for EA’s Spore.

Additional Resources

Write for UX Booth

Contribute to UX Booth
Contribute

Contribute a guest post to UX Booth and let the community know what's important to you!



Comments

  1. Adam Bitner February 16, 2010

    I’d love a copy of Remote Research! We do remote usability testing at my company and would certainly like to read tips from the gurus.

    Twitter ID: @apbitner

  2. Sounds like a great book, sign me up for a copy @suimple. For remote research I use TeamViewer to run the meeting and record and edit with Camtasia.

  3. Awesome interview, Andrew! Definitely want my hands on a copy. Username on Twitter is jamescosta :-)

  4. Great interview! I saw Nate speak at SxSW and was looking forward to this particular book. In fact, I have several books from Rosenfeld Media and would recommend them all, especially “Web Form Design” by Luke Wroblewski.

  5. Damn, forgot to include my twitter acct:

    @brucefloyd

    Thanks!

  6. Very interesting.. great taster! @yaellevey

  7. Will not play the “I’ve red all the article” guy. I’m reading it, but can’t miss my chance to be given a copy… So what is remote research ? I will find ou in a moment i hope…

  8. Hey Andrew thanks for posting this interview. I have been eyeballing this book for a bit now.

    I just started a new position at a larger company and research is a big part of it… I would love a copy of this book and could put it to some serious use.

    So when you are dishin’ out the love don’t forget to dish some to @aaroni268 :)

  9. Hey, thanks for sharing the interview and I would love to have a copy of the book, here is my Twitter username: @simpleasthat_

  10. Nice interview. I really enjoyed Nate’s presentation at Ix10. We do a lot of remote research, and even with all of our experience there were some great new tools and pointers.

    Would love to get a copy of the book for our departmental library…and one for myself.

    @dantemurphy

  11. We perform remote testing all the time, and I agree, we need to look further into more catch and release tactics. I hope I win the book but I think I will have to ask if we can buy it in our team, Thanks for the article!

  12. I’m feeling lucky. Sorry everyone!

    Twitter: @jeromebot

  13. Interesting interview, and it sounds like a great book. I especially appreciate the advice to “just do it” as someone who “isn’t treated as a first-class citizen” ha.

  14. Inspiring interview, especially “Intern somewhere that conducts UX and IX research”. I’m working on it. Definitely adding this book to my read list. Even better if I can win a copy from UXBooth :-)

    @calvincchan

  15. Interesting interview, good points there. I’ve been trying to do more usability testing at my company lately and doing it without asking others for permission really seems to be working. Hopefully people will start asking for it at some point.

    And because the book looks interesting and I’ve come here mainly to try my luck, here it is: @falkowata

  16. Thanks for preaching the gospel of Usability and Usability testing! I can’t wait to read the new book.

    @travisbsmith

  17. You’re giving away a book? Sure, I’m very interested in reading this.

    • Hey Marko, if you wanna win, you’ve gotta enter! Leave us your twitter ID and follow @uxbooth. Thanks for your interest.

  18. More testing! More Books!

    @introworks

  19. Blair Keen February 21, 2010

    I would love a copy of your book.

    Twitter: @AdditveUK

  20. Nice! Looking forward to reading it!

    @colemandash

  21. Thanks guys for the great opportunity to win this amazing book

    twitter.com/sdarine

  22. You guys really have helped me get into UX – it’s recently become a major aspect of my job and I am thankful for your constant updates.

    3 cheers!

    @duckyxxx

  23. Sam Sherwood February 22, 2010

    Look forward to giving this a read.

    @ruddyseal

  24. Sounds really thoughtful !!! Should be the second book of UX I would read after Kuniavsky. Notify to my twitter ID @rvkmrj for a book copy

  25. Very cool – would love to win that book! cheers! @optikinescant

  26. We do a fair amount of user research, but very little of it is remote. Would love to get my hands on a copy of this book!

    @sachopra

  27. I want that book :D

    @vladgeorgescu

  28. Břeťa Passinger February 22, 2010

    I got from Rosenfeld Media @ books (in PDF) – Web Form Design by Luke Wroblewski and Mental Models by Indy Young.
    Both worth the money spent.

    And I do dare to tweet about that @BretaPassinger

  29. Really interesting interview. Would love to get a copy of the book.

    @mspownall

  30. Absolutely agree with the focus on “real users use real products with their “natural” environments”.

    If you want womething to work in the ‘real world’ then assess and design it in the real world.

    tw: @Emma_Pickering

  31. Thanks for the interesting blog and for the book :) tw: @imadmouaddine

  32. Tony Collen February 22, 2010

    This book will be a great resource — not only are our customers and users remote to us, but I also telecommute.

    – @tcollen

  33. Great article and I would love to have a read of the book. It would be very useful for my work as a few of my clients are remotely located.

    @profb

  34. I’d love a copy! @debweb

  35. @drewshapter Certainly sounds like one that needs to be read

  36. I’d love a copy of this book. Own a couple of Rosenfeld books and they are always insightful reads.

    Twitter ID: @lorraine_p

  37. Patrick Lajeunesse February 22, 2010

    I’ve gotten so much value from guerilla testing… but I’m currently prepping for a bunch of remote testing sessions. So good timing. ;)

    @patlaj on twitter.

  38. Tim Thompson February 22, 2010

    Would love a book!

    @timothompson

  39. The winners, chosen by Random.org, are: @aaroni268 @apbitner @lorraine_p.

    Congratulations!

  40. Thanks Gang! I am super stoked for this book… cant wait to put it to some good use.

  41. @JessMarieHolt – very interesting article. Always the debate to go remote or not.

  42. Just read this post and realised I’m too late to ask for the free copy! It would have been timely – ‘Remote Research’ is the first title we’re reading at the new Birmingham UX Book Club (UK):

    http://uxbookclub.org/doku.php?id=birmingham

    Looks like a great title. Useful insight too in this post on just doing it and where to find the tools to start.

  43. Thanks for the discount code Matthew. Much appreciated!

Related Posts