UX Booth

The UX Panel and Usability Review Procedure

Author Avatar

Author

Redd Horrocks

Published

March 12th, 2009

Popularity

Share

Category

Blog, News & Events

Related Topics

10 Comments

Since this site launched in November of 2008, the UX Panel has done nine full free usability reviews of submitted sites across a broad range of categories.

David Travis of User Focus left us this comment on our last Usability Review:

“Your reviewers are a theatre administrator, a student, a blogger and a developer. Although their comments are interesting, what qualifications do they have to review a site’s usability? To achieve objectivity, usability reviews should be structured and formalised, otherwise you end up with a biased, subjective user review, not a usability review.”

This is a fair comment. What does qualify us to be Usability Experts? How are our review processes structured? Well, here is a little more information about each of us and our backgrounds, beyond what you see in our bios.

Who We Are

First and foremost, what qualifies us to give objective and well-rounded usability reviews is that we are all different and we are all normal people. We have a very diverse set of expertise and skills which allow us to all approach reviews with different mindsets, desires and pet peeves. We like very different things.

Redd Horrocks

Redd is a part time theatre administrator and techie. But she also works for MailChimp, one of the premier email marketing companies, in their technical support department. She is also highly internet literate, a strong researcher and writer, and a customer service expert. She is responsible for several other blogs and she created and sold her first online company at the age of 24, and is now working on her second. She also founded the Atlanta Bloggers Meetup group which currently has over 150 members. She is great with people, and rarely meets someone she doesn’t get along with.

Redd is a common web user. She represents non-coders and standard users.

Matthew Kammerer

Matt is currently a student at Young Harris University studying Education, Business and Public Policy. He has built several successful websites as well as worked in eCommerce since he was a teenager. He is also the Assistant Site Manager of BuySellAds, and works heavily on their Marketing and PR. Matt is great at integrating UX Booth into the community and does a lot of our PR. He is also a leader at his University, being heavily involved in his university Student Government and is currently serving as President.

Matt is another more common user. He represents marketing and students at the same time.

David Leggett

David is a blogger to a certain degree. What he really does though, is build and run incredibly successful sites. He is a competent designer and knows what makes websites tick. He’s an expert at generating traffic and revenue and he’s been doing this for a really long time. David is an accomplished and experienced public speaker and consultant. He is a businessman, through and through.

David is a power-user. He represents business people and experienced bloggers and site owners.

Andrew Maier

Andrew is a developer, that’s true. He is also a designer and photographer. He graduated with High Honors from the Georgia Institute of Technology with a degree in Discrete Mathematics. He now works for Hashrocket, arguably one of the best Ruby on Rails development firms in the nation. He’s a great teacher, and taught computer science at the Governors Honors Program in Valdosta, GA in 2006.

Andrew did the design for UX Booth, and he has also redesigned several other websites, focusing on keeping them user friendly for the user and the owner. He is experience in content management and creating clean code. He’s also a fantastic photographer with an eye for color and composition. He builds beautiful things and wants to make the world prettier through web design.

Andrew is a power-user. He represents designers and developers.

How We Do Our Reviews

We have a very standard procedure for how we do our reviews.

The sites that we review are submitted by the community. We then narrow down the list to 10 different sites, and the community votes on what they want to see reviewed. The top six are then scheduled for review.

When the UX Panel reviews a site, we each work independently and build our review in different ways. Some of us will record what we do in Silverback and then transcribe everything later, some of us write the review directly from our first time around the site.

We work our way through the entire site, checking for things that do and don’t work, things we like, things that confuse us. We look for ways of making the site more user-friendly and give what recommendations we can to that effect.

Because the four of us come from such different backgrounds and have such different skills, we tend to look for different things. Andrew and David tend to look at all of the very in depth technical aspects of a site, while Matt and Redd tend to look at things like how easy it is to navigate and whether you can find the contact forms. They look at how user friendly the site is for the layman.

Each member of the UX Panel reviews a site individually first. A composite review is put together afterwards

After we have all written up our full reviews, they are published on the site and the composite review with snippets from each of us is put together. This has a very small amount of information in it compared to the full reviews, with one snippet from each of us per category.

These composite reviews are what you see on our site, with the links to the full review below.

Doing it this way, where we all do our usability reviews independent from each other allows us to all voice our own opinion without having our minds changed or influenced by other members of the panel. We all know what we like and don’t like, and nine times out of ten, we all agree. We can give four whole, complete opinions from people who are very different to each other and work in different fields.

When testing usability, we believe that it’s important to look at it from a users point of view. Yes, we don’t necessarily have years of college training specifically on usability, but we are users ourselves and we’re sharing what works and what doesn’t work for us.