What is a heuristic evaluation?

Heuristic evaluations usually are conducted by one to three evaluators. The evaluators independently examine a user interface and judge its compliance with a set of usability principles.

The result of this analysis is a list of potential usability issues or problems. The usability principles, also referred to as usability heuristics, are taken from published lists. Ideally, each potential usability problem is assigned to one or more heuristics to help facilitate fixing the problem. As more evaluators are involved, more true problems are found.

Some evaluators try to estimate the degree to which each usability issue potentially could impede user performance or acceptance. This is done to help set priorities for making revisions to the system. However, judging the severity of proposed usability issues has been shown to be very difficult to do.

What are the advantages of a heuristic review?

The method can provide some quick and relatively inexpensive feedback to designers. Feedback can be obtained early in the design process. Assigning the correct heuristic can help suggest the best corrective measures to designers.

What are the disadvantages of a heuristic review?

Using this type of evaluation to identify usability problems is very difficult, research shows that it may result in only a 50% hit rate, and a 20% miss rate when evaluated by one evaluator.

Results are not based on observations of user behavior, and will result in suggesting changes to the website that are not required (about a 50% false positive rate). If the wrong heuristics are assigned to potential problems, it will mislead designers into applying the wrong solutions to the problems.

How many evaluators are needed?

Recent research indicates that when you increase the number of evaluators increases the hit rate by 15%-20%. In order to maintain the proper cost-benefit ratio two or three evaluators is recommended.

As an recommendation if you are operating an e-commerce product website use an usability specialist who has experience in operating an e-commerce product website. A evaluator will no experience will find 22% to 29% of usability problems.