Automatic usability monitoring

Wouldn't usability be easier if you could just fold it into web analytics? Your web analytics vendor thinks so. Companies like Tealeaf are selling add-ons that catch errors and watch metrics to find usability problems with your website. But web analytics is not real usability.

First of all, the usability best practice is to catch problems before they happen, through good user-centered design and usability testing of prototypes. Analytics are only good for finding usability problems after they've bothered your users.

Second, an analytics package can only do half the work. It'll tell you something is wrong, but in usability it's even more challenging to know how to fix a problem. Let's say you publish an online newsletter, and too many users are submitting incomplete signup forms. There could be all kinds of reasons for this: misleading language, an overly lengthly form, or an error in the form code, to name a few. Tweak the wrong thing, and you actually could worsen performance.

Third, when it comes to usability, it's easy to misinterpret statistics. If you update your product design and users begin taking longer to accomplish tasks, is that really a problem? It could be they're being more careful and getting better results. Perhaps it's a temporary change that will fade as users grow accustomed to the new look and feel. Maybe they're watching that new video advertisement you implemented.

I'm very skeptical of automatic usability monitoring, though I expect to encounter it more and more over time. After all, usability experts are expensive, and the technology industry has a natural bias  towards technical solutions.

Comments

[James Breeze]

What other ones have you seen?

[Joshua]

Besides Tealeaf, there's CrazyEgg.com, and one other I found when I wrote this entry, but I can't find it now. It's easier to find consultants who promise to do usability analysis based on your web analytics package.

Major analytics vendors like WebTrends and Omniture will find it irresistible to deploy similar modules. And many businesses will respond to "productizing" usability. It's tempting for all parties to cast usability as a simple problem that can be neatly quantified and solved at one stroke. But of course, usability is really a process or a way of thinking about customers.

More info:

http://www.boxesandarrows.com/view/web_traffic_analytics_and_user_experience

http://www.uie.com/brainsparks/2006/03/08/determining-usability-from-analytics/