Overcoming the Obstacles of Usability Testing

Usability testing, though incredibly useful, is not without its own hitches. Author and self-described "usability geek" Jacob Creech walks through the most common issues and how to avoid them.

When people hear about ‘usability testing,’ many things come to mind—eye-tracking cameras, big HCI labs, a long testing process, a lot of expenses, and maybe a little confusion as well. Even at this stage in the proverbial game, usability testing isn’t so well understood, and misconceptions abound.

While it’s true that when doing usability testing there are several stumbling blocks that can impede your progress (lack of funding, lack of time, lack of motivation, and especially lack of testers), there are also many ways to get around these issues. There may even be reasons why they’re no longer valid in today’s Internet world.

Today I will address these issues and give you some advice based on my own experience in conducting usability tests. With any luck, it will help you design and develop a more usable world.

High cost

In the past, usability testing meant getting people into labs, arranging testing sessions, utilizing eye-tracking cameras, and so on; clearly, high costs to go along with that. Times have changed, and the Internet has heralded a new generation of usability testing solutions. Some online services out there do charge a packet, there are other services out there that charge significantly less and are very affordable for any kind of project. The service I’m involved with (IntuitionHQ.com) only charges $US9 a test, which is affordable by most anyone’s standard. There are many low cost options out there, so price really doesn’t need to keep you away from testing your site, designs, and UI. In fact, investing in usability can generate a handsome return as you develop superior, more usable designs and the happier clients and users that come with that.

Costs vary based on requirements. There is something for everyone

Costs vary based on requirements. There is something for everyone.

“If the user can’t find it, it doesn’t exist.” — HFI Button

Lack of time

Time - use only as much as you need

With modern methods, you can get results in minutes, not months.

With regards to lack of time, this may have been a problem in the past. If you wanted to get people in for lab testing sessions, the time requirements were immense. The Internet has changed all that too, as there are all kinds of remote usability testing services out there, with a huge range of different features depending on your requirements. Depending on the feature sets you require, and time frame for your project, you can find a number of options to fulfill your needs. Again, looking at IntuitionHQ, setting up a test takes just a few minutes depending on the tasks you include, and you can easily get results back in no time.

There are many different kinds of remote testing solutions depending on your own unique requirements for testing. See what fits for you and run with it—you won’t regret it, and you’ll end up saving time by having less micro-management from clients.

“Pay attention to what users do, not what they say.” — Jakob Nielsen

Lack of motivation

There are usually several reasons for lack of motivation—people don’t think their clients or users care about usability, they think their designs already work flawlessly, they don’t understand the testing process, or they can’t see the benefits that come from testing. Well, it’s safe to say: if you’re reading this, you probably don’t hold this opinion, but bear in mind all of the positives that testing can produce:

  • Your designs can be validated;
  • Any potential usability issues can be picked up;
  • Your clients can be involved in the process without being over-involved;
  • Your clients can better see the value of your service;
  • You can avoid design-by-committee;
  • Testing can be sold as an added value service to clients and can help sell your service and designs.

These are just a few of the points to consider. Of course, there are many more reasons why you should be pro-usability testing. See my previous article for more on this.

“Design is so critical it should be on the agenda of every meeting in every single department.” — Tom Peters

Lack of testers

Testers are obviously crucial to conducting any usability test (I challenge you to conduct a test without them). There are two things to consider when finding testers.

Using the internet, you can get testers from around the world

Crowd source with the power of the Internet.

First is the oft quoted number from Jakob Nielsen about picking up 85% of usability issues with just 5 testers. There is a really great explanation of this at MeasuringUsability.com. The basic premise is that according to statistical analysis, a small number of users will pick up the vast majority of usability issues, and the cost for testing with more than 5 users isn’t worth it (at least generally speaking) to find the last 15% of usability issues. Well, this is obviously dependent on the kind of tests you are performing (and the costs for getting people to help you test), but it’s nice to keep some sort of number in mind so you know when you start receiving useful information.

Second is the testing method and timeframe for the project. If you were bringing people into a lab, 5 people may be enough. With heatmap tracking (à la IntuitionHQ), larger numbers will produce more concrete evidence of issues and help you to track them sooner. Some sites can also provide you with testers, but when looking at their methods I get back to both the cost issue and self-selecting samples again. It seems to me if you can control and distribute the test yourself, you will end up with a broader range of feedback from a wider audience. So, how to distribute your tests?

Distributing tests

Social media - the perfect medium for distributing tests.

Social media—the perfect medium for distributing tests.

Ok, with all that out of the way, and assuming you have your tests set up, the next step is getting it out to the people. How you choose to approach this may depend on the site you are designing and what the objectives are. These are some steps that have worked for us:

  1. Clients: Clients almost always want to see what they are paying for and to feel like they have at least some say in the design process. They are enthusiastic about the project (or at least they should be), and if you can involve them (especially without taking up too much of your time), you should do it. Send them tests and get their feedback—they are probably in the right demographic of the market for the site since they are the ones who are trying to sell it.

  2. Stakeholders: These can be investors of the sites, current users, or anyone else with an interest in the site. Again, they are the perfect demographic for finding issues with the site; after all, if they have difficulties, you know something probably needs changing. You can get them involved in a number of different ways—sending the link out via email or on the RSS feed, having a link on the body of the site (if redesigning), asking the client to distribute it to friends, family, or other people interested in the site (this of course depends on the kind of site, the client etc.)—you are only limited by your imagination.

  3. Social Media: In this day and age, they are so many people on Twitter, Facebook, LinkedIn et al that you’d have to be living in a Faraday cage to miss them. Make the most of this resource; get out there and share your tests with your friends and followers—people love to see what others are working on and to be able to contribute to the process. It generates talk around the brand too, demonstrating that they put extra thought into design and usability.

  4. Forums: Yes, forums do still exist, and in fact, a large number of people still frequent them. You can find a million and one different design and development forums, but you can also find ones more specific to the genre of the site you are testing (fashion forums for a fashion site, camera forums for camera sites, etc). Whatever the case may be, this is another great way to generate feedback.

  5. Surveys: We often see people who are redeveloping their sites run surveys to gather feedback. While you are pulling in this information, why not throw in a usability test as well? It’s just as easy to complete as a survey, and can tie in very nicely with other feedback you are pulling in.

To sum things up

Hopefully these suggestions will give you some ideas on why you should integrate usability testing into your design process, and how you can get it out there when you’re ready to go.

If you have any questions about usability testing, or this article, be sure to let me know in the comments section. I’m really happy to help if I can.

About the Author

Jacob Creech

Jacob is a usability geek at Boost New Media, the people behind IntuitionHQ.com - a funky, new, quick and easy to use usability service. When he's not indulging his passion for the internets, he is probably studying Chinese (having spent three years in China) or reading about design.

Related Articles

16 Comments

  • Tommyismyname Reply

    If there is one thing that can kill any sort of marketing efforts and sink a brands credibility online, it’s usability.

    Designing the site with the end users goals in mind in the first place is the best place to start.

    • Jacob Creech Reply

      Thanks very much for your comment Tommy – you are absolutely right, couldn’t have said it better myself!

      Cheers.

  • Joshua Lay Reply

    Another great article Jacob.

    I agree completely. Usability testing should be introduced as a component of the design process. Each step of the way testing it done.

    The overall benefits are so much better. It’s easier to mould clay when it’s wet than after it has been shoved in the kiln!

    Better usability, happier users, more users, more return on investment.

    • Jacob Creech Reply

      Thanks Joshua; well said.

      I think once people realise the payoff for them, they are much more likely to go ahead with testing. Fingers crossed that it will lead to a much more usable web for the rest of us!

  • Jurij Burkanov Reply

    Can’t say about the US, but in Germany usability tests are very rare thing. I’ve been working in Germany since 1998 in all kind of web-agencies, made, probably, more than 100 websites, some are really large (like the ones for ministries, or tv-companies, e.t.c.). There were not any usability tests ever. More than that, no one of any of my coworkers have ever worked for any company, that has ever done a usability test.

    In internet you can read ten million articles about cheap usability test, fast usability test, how important they are and so on. But the truth (at least for Germany) is: if you would ever mention to your client, that you’d like to make a usability test – you’re going to loose your client. I understand, that it sound all wrong and perverse, but: doesn’t matter how cheap it is, people tend to see a NEED for usability test as a involuntary admission in incompetence. Like, “you need this test only because you don’t know how to make my website cool without it. So I better find someone who’s experienced enough to make websites without learning some bullshit at our cost”. Exactly the same way it works inside: if you, as web-developer would propose to make a usability test to your manager – the reaction, at last, will be the same.

    All of this articles about usability tests lack one most important argument: how can you explain someone, that usability test means high professional quality and not the otherwise? Just imagine someone client who has to choose between company “A” and “B”. Both say, they can do the job done, but “A” says they need some extra money and time to… well… find how to make this site user-friendly, and “B” says, they got such brilliant designers, that they can do anything user-friendly just like that, without extra “research”.

    As a result of all this politics in Germany:
    – “B” always (not in 99%, but in 100%) wins
    – Usability testing is studied in the schools, but never practiced
    – Almost all german websites are way worse in usability, IA and ROI then american ones.

    • Jacob Creech Reply

      Thanks a lot for your comment. I can’t say I’m that familiar with the German market, but it’s certainly interesting to get some insight into it.

      It seems to me, if someone was going to usability test, that means they are almost guaranteed to develop a better site. Not choosing them because of this aspect seems strange to me.

      Perhaps you could include charges for testing, and simply explain it as part of the design process – which it really is, and that way you can still ensure you develop more usable designs.

      There are a couple of interesting articles at http://www.usefulusability.com/7-reasons-why-you-can’t-sell-usability/ and http://www.userfocus.co.uk/articles/sellingusability.html which might be of use to you.

      The core argument is making sure that you educate people as to the value of usability testing, and then once they see the value they won’t be so afraid of it.

      Thanks again for sharing, really interesting to know all that.

    • Adam Hermsdorfer Reply

      No kidding…there are a ton of great, cost efficient usability tools online right now. Ranging from feedback army, userfly, usertesting.com, etc.

    • Jacob Creech Reply

      Glad you said nice ad, wouldn’t want a bad one.

      In all fairness though, all of these points apply to a number of remote usability testing services, it just happens that IntuitionHQ is the one I’m most familiar with.

      Hopefully there is some useful information here that you can apply to which ever remote testing service you prefer.

      Cheers.

  • هتل Reply

    Thanks very much for your comment Tommy – you are absolutely right, couldn’t have said it better myself!

    Cheers.

Leave a Comment on This Article