Userfly: Usability Testing Made Easy

Userfly – Web Usability Testing Made Easy

Userfly is described as usability testing made easy. Since my review of Feedback Army I have been overwhelmed with different websites offering usability services in a quick and easy package. However, Userfly seems to stand out from the crowd! I have put it to the test and brought you my findings.

After using Userfly I can see why so many people talk about it. This application is thoroughly thought out and professionally done. To top it off, they even offer 10 free recordings a month to see if the service is right for you. How could I resist the temptation of testing such a service?

How it works

Userfly describes it’s service as the following:

userfly.com provides instantaneous web user studies by recording user visits and letting you play them back to see every mouse movement, click, and form interaction. Conducting a user study doesn’t have to be expensive or a logistic nightmare. With userfly.com you can perform simple and cheap user testing with your real users. And it only takes one minute to set up!

Userfly Code SnippetUserfly operates through a piece of javascript you place in your head tag.

I would say this is a pretty straight forward description of the service that I found to be true. This service allows you to easily record your visitors and reports are offered you to in video format. This report also contains many other helpful features and options.

Using Userfly to record your users is only the first step, however. Let me give you some insight on how to use this to better your website.

Steps to learning from Userfly

Following these steps can help you understand your user’s interactions with your website. Learning from these actions can help you improve UX by watching the movements within your user interface.

  1. Embed the Code

    Once you sign up you are brought to offered a tab labeled Install. This option will present you with the javascript you asked to place in your head tag. There is even an option for one click install for TypePad and Blogger users.

    Be sure to check your site after you’ve installed the javascript. We had a slight mishap at UX Booth when entering the javascript. It seems that the script adds IDs to elements that don’t have IDs. Now I am not a big coder so I can not go into detail but Andrew and I looked into it a bit and discovered the CSS for our headers did not agree with having an ID added to them. For a short period of time our headers were whacked out and we had to remove the code. I continued testing on my gecko blog that I run and had no errors there.

    Also, be sure to remove the code when you want to stop testing. It seems there is a small bug at the moment that records the occastional visitor even when you specify a pause in the session recording in the Userfly control panel.

  2. Wait for Users

    Once the javascript is installed the recordings start rolling in. Be patient, you will want to wait an hour or two if your website does not have high traffic to attain some real valuable recordings! Running Userfly on my gecko blog, which averages only between 100 to 200 visitors per day, I got some great results in the first 10 to 15 minutes but the real eye opening recordings happened in the next few hours.

    Also notice that Userfly by default will record you as a visitor too. You do have the option to block IPs through the control panel. You may consider doing this for your IP. Something else I noticed is that recordings that are very short are not added to your listings to prevent wasted time and money. I really liked seeing this!

  3. Understand the Reports

    Each aspect of the Recordings page is important. The first portion of the Recordings page tells you what page was recorded. This is important to take note of as it is the starting point at which you viewer has landed on. Various parts of your website may have different templates and structures and it is important to take note of how possible first time visitors react with exposure to each type of page.

    The next part of the report shows you exactly where the user came from, if it can be detected. I saw an interesting difference in visitors from different sources. Some of our readers that visited from PSD Tuts stuck around and read articles in depth, while visitors from social websites came and skimmed very quickly. More research is needed to understand if this is valid pattern but it is something to consider when trying to keep users around for longer.

    The next two features are date, as in when the recording was captured, and page views, which tells you how many pages were viewed in the visit and how long the user was there. Both of these things can be factored into a bigger picture to see how different influences may change your users duration and exploration.

    Next is the location of the visitor with an option to view this location on a map. Users from different countries or even regions may interact with your site differently. Finally you are presented the browser the user is viewing you with as well as the option to save, delete, or ban an IP.

  4. Watch the Videos

    Watching the videos is also very simple process. Simply select a visit and enjoy.

    This part is the most important bit, really. You need to set aside some time to sit and watch these. They can sometimes be boring and often time consuming. Some recordings it seems like the user fell asleep at the mouse when reading. Just remember, these interactions are what will lead you to your improvements!

  5. Take Note of Interactions

    This really is the epiphany of Userfly. Pay special attention when watching those interaction recordings.

    Are users from search engines visiting your website and immediately leaving through google ads? Is it worth the loss? Do users from a specific forum seem to stay longer? Would you consider advertising with them to increase a stable audience?

    Ask yourself these kind of questions. Also, pay attention to what people click on and how they navigate. You know your design, you think it works well. You may be surprised when people click on items you never thought would look like links or not understand how to easily return to the home page. Studying these interactions can build a data bank of knowledge that will increase your UX skills and bring you up that extra notch you need!

  6. Correct Problems

    This is it, time to bring it home. Now that you’ve amassed a list of problems that your website showcases you can now correct them! Use what you’ve learned and apply it to make the user experience better and more enjoyable for your users. It will easily increase profits, subscribers, and viewers.

    Don’t stop now though. Test again, improve again, and repeat! The fight is a never ending one for people interested in improving usability.

What I liked

I think I’ve made it clear enough that I feel that Userfly is well thought out and covers the bases. Let me elaborate a bit:

Userfly shows where the visitor comes from in the Refer section.

  • The referral link: Having the referral link as part of the feedback process allows the user to see how visitors from different sources operate differently. In my testing it is clear how someone from a forum navigates differently from a search engine. This information can lead to a better experience for the users and create better sales, more loyal visitors, and a happier viewing experience.
  • Video is creepy at first, but very useful: Ok, at first I was a little creeped out watching my viewers as if it were live. After getting over that I was amazed. People were clicking on things that weren’t links and reading pages I expected people to only quickly scan over. Overall it gave me a clear understanding of how my user interface worked, and more importantly didn’t work.
  • Userfly IPThe ability to block IPs can be useful.
  • Ability to ignore IPs: I was happy to be able to ignore IPs. This feature allows me to block recordings of my visits, as well as IPs that you find not creating valuable feedback for you. It’s all about saving resources for those recordings that really count!
  • Smart recording, to save me money: Something I really found useful was the fact that Userfly does not parse visits that are very small or insignificant. If I pay for 20 recordings I do not want a third of these visits to be of someone landing on my page and exiting in 2 seconds. (Even though this could tell me something..).

What I’d Like to See Different

There are some features that I would like to see a little different, or maybe even some things added to this service:

  • Auto recognition of my IP: I think it may be useful to auto ignore the site owner’s IP address. This could save time for the tester. However, a simple note saying that your IP is being blocked could work as a preventative to any confusion of visits not being recorded, as well as an option to disable this for those who do not prefer it.
  • Charts

    Charts such as Google Analytics provides could be useful.

  • Lets see some charts: It’d be useful to see some graphical data. I am a visual man! I could see benefit in charting page view averages based on browser, referral link, location, and more. This data could also be used in presentations given to clients after using this service.
  • I’d love to keep these videos: I would like to be able to keep the video recordings of the users interactions for longer than you save them on Userfly. Offer an option to download an archive of the videos recorded. This could be useful for UX professionals to offer their clients along with the report of their findings.

Conclusion

The service that Userfly offers is simple: the ability to see exactly what your viewers see and do. This information can lead to the understanding of common user experience problems that in turn can be fixed.

I would highly recommend adding Userfly to your arsenal of UX related resources. This real life user testing is simple, fast, and helpful.

What do you think

Have you used Userfly? What aspects of the service were your favorite and what would you like to see changed or added? Leave us a comment and tell us below!

Write for UX Booth

Contribute to UX Booth
Contribute

Contribute a guest post to UX Booth and let the community know what's important to you!



Comments

  1. great review, and what an awesome service!

  2. See like a very useful tool, I might look into ivnesting in this, but I think i might wait until they make the video’s downloadable.

  3. I signed up for Userfly when I saw you mention it on Twitter. At first I was like “There’s no way this is a free service.” That first impression was correct. I would have liked to see pricing info before closing the deal. However, the bad taste that experience left in my mouth was quickly washed away by the elements of the service that are free, the very reasonable pricing on additional recordings, and the value of the data.

    I’m a big believer in small, iterative, talk aloud usability testing with “real” people. But I can see this service being great for follow up testing after a problem has been identified and addressed. Documenting successful completion of a task previously identified as a problem area could free up the next round of in-person testing to focus on unexplored areas.

    The problem I have with the service is the lack of context for explaining problems and failures. That can always be addressed through targeted tests in the future. But I think the investment of time in watching the recordings paired with the lack of feedback or context (in the way of thought processes involved anyway) make it a poor tool for the discovery of new problems.

    I’m not complaining. I’m always happy to add another tool to my UX toolbox, especially when they are cheap or free! :)

  4. We’ve been using Userfly, and we find it both addicting and frustrating. It’s fascinating to see what users are doing on the site, but we find it raises more questions than it answers. Who is the user? What is his or her goal? Did the information provided meet expectations? What is the user’s next step after leaving the site?

    Userfly may be most effective as a tool for generating hypotheses about usability issues that can be further explored in usability testing, where we can understand user goals and behavior in more detail.

  5. @Kathi Kaiser: Using Userfly in conjunction with a service like Loop11 to ask specific questions and tasks may be a good solution for the problem you present. :)

  6. Wow, that is pretty cool. I’ll have to look into it.

  7. Sergei May 19, 2009

    Use coupon code “TWITTER” for 100 free recordings.

  8. I am already in love with this.

  9. I wish I could get an ability to download recordings as avi or something.

  10. So, I had checked out Userfly a while back, but never installed it. After seeing your tweet, I went back to the site. Registering and copying and pasting the script took about 4-5 minutes total, which is awesome. They make it very easy to get started. This is a great example of “walking the walk” in my opinion, from a company designing to improve usability!

    In seeing the videos, I am very impressed. I don’t know how they pull it off, but it creates a high-quality recording of the mouse movements on the screen and highlights each click-point with a red dot.

    I was watching visits to my conference site, http://frontenddesignconference.com ; it has jQuery toggles, twitter links and a slider. Everything in the site functions in the videos and I am able to see how the visitors interact with the areas. I was able to see patterns of behavior and flaws in my design within the first 5 recordings.

    For example, nearly every visitor scrolls to the bottom of the page and then works back up to the navigation on the right. They click through areas of interest and almost always go back to one of the toggled areas.

    Ironically, the users toggle each area to open it and then close it before going to the next section; then, some return back to reread it. It’s not designed as an accordion, so they can have multiple areas open if they choose.

    I learned that I should have the Hilton name linked in the Locations section, because I could tell that a user was trying to click the name or copied and pasted it.

    Those are just a few of the things that I have seen in the videos so far. I really enjoy this and I will absolutely add it as a feature for future clients of mine. I hope to take the freelancing plunge before long…

    The key things to remember are that it is simple and easy to register and for quality results, use Firefox. (Chrome misses a lot of the javascript functionality in the playbacks. I only used it in those two browsers. I tried it in Chrome at first and was disappointed. I happened to be on a different machine later and checked the videos using Firefox and was able to see everything that the user did. That’s when I was wowed!)

    Great post Matthew!

  11. I used it before. It was very slow to load at the time. I used ClickTale instead. I still think it’s better. Maybe try the new version of UserFly.

    Anyone did research on speed when using UserFly?

  12. Everything about userfly is great, even the pricing structure!

    We can get the data from analytics, but actually seeing someone land on your site, scroll up and down and then leave really pushes the message home about skimming and scanning.

    I found that when readers stay on a page they don’t read in a linear fashion. It’s really quite odd, and just reinforces all that Nielsen says about structuring and emphasising information.

    I’ve used the info in the design of my site; simple things appending a list of related links to articles.

  13. You can get your data charts, web browser percentages, etc from basic google analytics, typically modern hosting packages include charted and measured browser data. Coupling something like this and mint, google analytics or your hosting providers built in tools should fill your voids.

  14. This seems really great. Bookmarked for future use.

    Thanks!

  15. Great review as always. Hello to UX Booth from Bangladesh! This is my first comment on UB, even though I’ve been a reader for ages…

  16. @Sergei: Nice, thanks!

    @Tejas: I also pointed that out :).

    @Mike van Hoenselaar: I did not see a big problem with speed

    @Dan Denney: Thanks for your awesome input!

    @Rolam Shahin Designs: Well I am glad you finally said hello, thanks for reading!

  17. Dieter Davis May 22, 2009

    Interesting tool – gets people introduced to replaying their user’s sessions. I’ve seen a number of tools that do similar, and actively use one for enterprise/commercial use (Tealeaf).

    In just reading this review, some things that came to mind on making design/usability decisions off of Userfly:
    A. Need to record all sessions (or a big chunk). If someone does a certain action (a problem), you need to be able to quantify how many users it impacts. Does everyone have this problem, or only 5 of 1,000?
    B. Finding a problem may be a bit of a lottery. That could be a lot of time just replaying sessions. You could hit upon the big or most frequent issues, but without #A, how will you know that’s what you need to focus on?
    C. In just reading the review, I’m not sure if it’s possible, but if you have a store website you might have to be careful about capturing secure (does this work with SSL?), or even sensitive payment information (gah!).

    But then, as others commented – for blogs or cloud-hosted products this might be great for getting individual insights, instead of aggregate info from Google Analytics. A heads up though on the above, if you need to scale.

    PS – On the “creepy part” (What I Liked section), I remember this from when I took sales calls for an internet website. When I spoke with older customers (50+), they might occasionally ask “Well, can’t you just see my shopping cart?”. I thought this seemed reasonable logic.. They see video cameras in stores, and always hear “this call may be recorded” on every phone call to a business. Food for thought…

  18. Thila May 25, 2009

    Another gimmick, another day. Any SEO professional with real experience can get detailed user-metrics by simply doing their job – properly.

  19. You can get your data charts, web browser percentages, etc from basic google analytics, typically modern hosting packages include charted and measured browser data. Coupling something like this and mint, google analytics or your hosting providers built in tools should fill your voids.

  20. I like being able to see all of the movements and clicks my users make but doing it one by one is really a waste of time. It satisfies my curiosity but by the time I come to any conclusions I have neglected so many other things that could have been much more productive.

    I find heat maps and the like to be much more user/time friendly.

  21. Oops, nevermind my previous question. I found out how to exclude my ip. My IP wasn’t recorded at the time I asked the question. Aparently you have to be captured before you can exclude it. Thanks again for the review though, very useful!

  22. I’ve have tried to contact their support for over a month now, without any kind of feedback from them.

    They are withdrawing money from my account, even though I’m on a free plan. The reason why they can withdraw money from me, is that I for a month where going with a paid solution and afterwards was going back to the free plan. It don’t work that work, I guess. They have charged me 2 month now and I’m talking with my bank so they can stop the transfers.

    They have a good product, but don’t buy anything from them or you are stuck paying them forever, or maybe I’m just unlucky.

    Kind regards,
    Niels

  23. Haven’t yet tried it myself, but if it’s javascript, I’m assuming if js is disabled no video/data captures will occur, am I right?

  24. Evelina March 24, 2010

    I found another great service at http://ghost.eemt.se Better than userfly, and alot cheeper. It works on user logged in sites too!

  25. An I the only one that finds this creepy and invasive on the users’ side? The idea that any website you go to can log your mouse inputs (and probably keyboard too?) without you knowing about it bothers me. I completely understand the need for usability testing, but recording your user’s actions without letting them know seems questionable to me.

    If it included a pop-up or something asking if they minded participating in a usability study, that would be perfect. But without that I will have to make sure I add userfly.com to my script block list.

  26. @ Bill websites are really private spaces, aren’t they? as much as it feels like we’re moving through the public domain at our own discretion we are actually visiting, the majority of the time, copyrighted, branded “sites”. grocery stores use security cameras to track customer behavior throughout their store and, for good or evil, use the resultant data to manipulate or enhance their customers’ experience. website owners are doing the same in this case.

  27. Think twice, before go with userfly, they record your credit card details, in so called “secure and safe” page

Related Posts